[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:58:45 dist-test-slave-6nmv krb5kdc[30156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:58:45 dist-test-slave-6nmv krb5kdc[30156](info): set up 2 sockets
Aug 12 18:58:45 dist-test-slave-6nmv krb5kdc[30156](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 18:58:46.131726 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:58:46 dist-test-slave-6nmv krb5kdc[30156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:58:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:58:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:58:46.246536 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:33521
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:33521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:46.347304 30171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:46.347570 30171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:46.347642 30171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:46.351495 30171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:58:46.351575 30171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:46.351626 30171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:46.351702 30171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:58:46.351737 30171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:58:46.356703 30171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:33521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:33521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:46.357692 30171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:46.358690 30171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:46.367198 30178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:46.367187 30176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:46.367187 30181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:46.367406 30171 server_base.cc:1034] running on GCE node
I20260812 18:58:46.368060 30171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:46.369059 30171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:46.370239 30171 hybrid_clock.cc:648] HybridClock initialized: now 1786561126370221 us; error 90 us; skew 500 ppm
Aug 12 18:58:46 dist-test-slave-6nmv krb5kdc[30156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561126, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:46.372967 30171 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:58:46.373898 30171 webserver.cc:460] Webserver started at http://127.29.112.254:40221/ using document root <none> and password file <none>
I20260812 18:58:46.374432 30171 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:46.374524 30171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:46.374760 30171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:46.376760 30171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "acd170c077e3408fb7c5edb6f6da60f7"
format_stamp: "Formatted at 2026-08-12 18:58:46 on dist-test-slave-6nmv"
server_key: "6a0d398bd043453f615afbe95822bb4a"
server_key_iv: "5bf125453bc4f86bba6be7e6742a44dd"
server_key_version: "encryptionkey@0"
I20260812 18:58:46.377331 30171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "acd170c077e3408fb7c5edb6f6da60f7"
format_stamp: "Formatted at 2026-08-12 18:58:46 on dist-test-slave-6nmv"
server_key: "6a0d398bd043453f615afbe95822bb4a"
server_key_iv: "5bf125453bc4f86bba6be7e6742a44dd"
server_key_version: "encryptionkey@0"
I20260812 18:58:46.380977 30171 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 18:58:46.383369 30190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:46.384549 30171 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:58:46.384683 30171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "acd170c077e3408fb7c5edb6f6da60f7"
format_stamp: "Formatted at 2026-08-12 18:58:46 on dist-test-slave-6nmv"
server_key: "6a0d398bd043453f615afbe95822bb4a"
server_key_iv: "5bf125453bc4f86bba6be7e6742a44dd"
server_key_version: "encryptionkey@0"
I20260812 18:58:46.384789 30171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:46.401268 30171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:46.404009 30171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:46.404212 30171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:46.410851 30171 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:33521
I20260812 18:58:46.410866 30254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:33521 every 8 connection(s)
I20260812 18:58:46.411748 30171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:58:46.412490 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 30171
I20260812 18:58:46.412632 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:58:46.413910 30147 external_mini_cluster.cc:1428] Setting key 402713a1fa696f154b70d1c372089160
I20260812 18:58:46.414582 30255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:46.421058 30255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7: Bootstrap starting.
I20260812 18:58:46.423467 30255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:46.424379 30255 log.cc:824] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:46.426308 30255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7: No bootstrap required, opened a new log
I20260812 18:58:46.430428 30255 raft_consensus.cc:348] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd170c077e3408fb7c5edb6f6da60f7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33521 } }
I20260812 18:58:46.430675 30255 raft_consensus.cc:374] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:46.430739 30255 raft_consensus.cc:729] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acd170c077e3408fb7c5edb6f6da60f7, State: Initialized, Role: FOLLOWER
I20260812 18:58:46.431213 30255 consensus_queue.cc:260] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd170c077e3408fb7c5edb6f6da60f7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33521 } }
I20260812 18:58:46.431367 30255 raft_consensus.cc:388] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:58:46.431447 30255 raft_consensus.cc:482] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:58:46.431555 30255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 18:58:46 dist-test-slave-6nmv krb5kdc[30156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:46.432369 30255 raft_consensus.cc:504] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd170c077e3408fb7c5edb6f6da60f7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33521 } }
I20260812 18:58:46.432672 30255 leader_election.cc:302] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acd170c077e3408fb7c5edb6f6da60f7; no voters:
I20260812 18:58:46.432956 30255 leader_election.cc:288] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:58:46.433027 30260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:46.433272 30260 raft_consensus.cc:686] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [term 1 LEADER]: Becoming Leader. State: Replica: acd170c077e3408fb7c5edb6f6da60f7, State: Running, Role: LEADER
I20260812 18:58:46.433555 30260 consensus_queue.cc:237] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd170c077e3408fb7c5edb6f6da60f7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33521 } }
I20260812 18:58:46.434195 30255 sys_catalog.cc:564] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:58:46.434689 30260 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acd170c077e3408fb7c5edb6f6da60f7. Latest consensus state: current_term: 1 leader_uuid: "acd170c077e3408fb7c5edb6f6da60f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd170c077e3408fb7c5edb6f6da60f7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33521 } } }
I20260812 18:58:46.434787 30260 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [sys.catalog]: This master's current role is: LEADER
I20260812 18:58:46.435726 30261 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acd170c077e3408fb7c5edb6f6da60f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd170c077e3408fb7c5edb6f6da60f7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33521 } } }
I20260812 18:58:46.435824 30261 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7 [sys.catalog]: This master's current role is: LEADER
I20260812 18:58:46.438791 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:46.416881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.29.112.254:33521)
0812 18:58:46.417347 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:46.417365 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:46.418408 (+ 1043us) server_negotiation.cc:407] Connection header received
0812 18:58:46.422185 (+ 3777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:46.422199 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:46.422472 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:46.422695 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:46.425534 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:46.426550 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:46.428022 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:46.428454 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:46.428562 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:46.432993 (+ 4431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:46.432997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:46.433024 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:46.433056 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:46.435192 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:46.435757 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:46.435764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:46.435769 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:46.435907 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:46.436367 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:46.436370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:46.436372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:46.436922 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:46.437113 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:46.437614 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 18:58:46.437918 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":145,"threads_started":1}
W20260812 18:58:46.439071 30280 catalog_manager.cc:1558] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 18:58:46.439160 30280 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 18:58:46.439386 30267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:58:46.440186 30267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:58:46.444612 30267 catalog_manager.cc:1347] Generated new cluster ID: 819c68e6098e4cd3b987402b60e9a199
I20260812 18:58:46.444667 30267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:58:46.449508 30267 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:58:46.450583 30267 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:58:46.458158 30267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acd170c077e3408fb7c5edb6f6da60f7: Generated new TSK 0
I20260812 18:58:46.458693 30267 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 18:58:46.488166 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33521
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:46.594101 30292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:46.594370 30292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:46.594415 30292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:46.598150 30292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:46.598206 30292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:46.598331 30292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:58:46.603277 30292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:46.604661 30292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:46.605769 30292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:46.616050 30301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:46.616055 30299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:46.616153 30298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:46.616338 30292 server_base.cc:1034] running on GCE node
I20260812 18:58:46.616726 30292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:46.617303 30292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:46.618461 30292 hybrid_clock.cc:648] HybridClock initialized: now 1786561126618443 us; error 29 us; skew 500 ppm
Aug 12 18:58:46 dist-test-slave-6nmv krb5kdc[30156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561126, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:46.621032 30292 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 18:58:46.621948 30292 webserver.cc:460] Webserver started at http://127.29.112.193:32979/ using document root <none> and password file <none>
I20260812 18:58:46.622462 30292 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:46.622525 30292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:46.622686 30292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:46.624632 30292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "3d70daf592674cbab7ef81f5b550ba9e"
format_stamp: "Formatted at 2026-08-12 18:58:46 on dist-test-slave-6nmv"
server_key: "d1242501ea9b57b41459df46101e47cb"
server_key_iv: "b7db42530fa94b288bc58eff435f64d7"
server_key_version: "encryptionkey@0"
I20260812 18:58:46.625150 30292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "3d70daf592674cbab7ef81f5b550ba9e"
format_stamp: "Formatted at 2026-08-12 18:58:46 on dist-test-slave-6nmv"
server_key: "d1242501ea9b57b41459df46101e47cb"
server_key_iv: "b7db42530fa94b288bc58eff435f64d7"
server_key_version: "encryptionkey@0"
I20260812 18:58:46.628808 30292 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 18:58:46.630997 30309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:46.632119 30292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:58:46.632208 30292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "3d70daf592674cbab7ef81f5b550ba9e"
format_stamp: "Formatted at 2026-08-12 18:58:46 on dist-test-slave-6nmv"
server_key: "d1242501ea9b57b41459df46101e47cb"
server_key_iv: "b7db42530fa94b288bc58eff435f64d7"
server_key_version: "encryptionkey@0"
I20260812 18:58:46.632287 30292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:46.655413 30292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:46.658664 30292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:46.658888 30292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:46.659509 30292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:58:46.660336 30292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:46.660406 30292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:46.660472 30292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:46.660506 30292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:46.669843 30292 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:38631
I20260812 18:58:46.669880 30447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:38631 every 8 connection(s)
I20260812 18:58:46.670662 30292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:58:46.674275 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 30292
I20260812 18:58:46.674391 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:58:46.674649 30147 external_mini_cluster.cc:1428] Setting key fb0e0f2bc0b17d9e3e73f56c3a346de1
Aug 12 18:58:46 dist-test-slave-6nmv krb5kdc[30156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561126, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 18:58:46.684116 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:46.672466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:51635 (local address 127.29.112.254:33521)
0812 18:58:46.672609 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:46.672613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:46.673618 (+ 1005us) server_negotiation.cc:407] Connection header received
0812 18:58:46.674620 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:46.674624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:46.674682 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:46.674753 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:46.676122 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:46.676639 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:46.677734 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:46.678040 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:46.678148 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:46.680715 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:46.680719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:46.680723 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:46.680749 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:46.682180 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:46.682668 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:46.682672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:46.682674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:46.682750 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:46.683064 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:46.683070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:46.683073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:46.683265 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:46.683374 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:46.683826 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 18:58:46.683951 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 18:58:46.684796 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:46.672809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33521 (local address 127.29.112.193:51635)
0812 18:58:46.673465 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:46.673504 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:46.674351 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:46.675003 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:46.675010 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:46.675466 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:46.675988 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:46.676005 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:46.676764 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:46.676768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:46.677607 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:46.677616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:46.678195 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:46.678199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:46.678358 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:46.678942 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:46.678959 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:46.680561 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:46.682293 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:46.682299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:46.682311 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:46.682557 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:46.682836 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:46.682841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:46.682842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:46.682975 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:46.683373 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:46.683379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:46.683603 (+ 224us) client_negotiation.cc:769] Sending connection context
0812 18:58:46.683785 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 18:58:46.684083 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":120,"threads_started":1}
I20260812 18:58:46.685940 30448 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33521
I20260812 18:58:46.686154 30448 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
I20260812 18:58:46.686677 30448 heartbeater.cc:511] Master 127.29.112.254:33521 requested a full tablet report, sending...
I20260812 18:58:46.688220 30214 ts_manager.cc:194] Registered new tserver with Master: 3d70daf592674cbab7ef81f5b550ba9e (127.29.112.193:38631)
I20260812 18:58:46.689774 30214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:51635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 18:58:46.702039 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33521
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:46.805850 30455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:46.806072 30455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:46.806169 30455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:46.809774 30455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:46.809870 30455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:46.809952 30455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:58:46.815531 30455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:46.816322 30455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:46.817267 30455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:46.826014 30463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:46.826110 30455 server_base.cc:1034] running on GCE node
W20260812 18:58:46.826071 30465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:46.826030 30462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:46.826809 30455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:46.827458 30455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:46.828624 30455 hybrid_clock.cc:648] HybridClock initialized: now 1786561126828610 us; error 48 us; skew 500 ppm
Aug 12 18:58:46 dist-test-slave-6nmv krb5kdc[30156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561126, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:46.831202 30455 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 18:58:46.832034 30455 webserver.cc:460] Webserver started at http://127.29.112.194:37605/ using document root <none> and password file <none>
I20260812 18:58:46.832567 30455 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:46.832664 30455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:46.832904 30455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:46.834988 30455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "4e6fa1c552d441ba8f03f88662a4ab11"
format_stamp: "Formatted at 2026-08-12 18:58:46 on dist-test-slave-6nmv"
server_key: "36456eb26519a0aba2da177738aa7cc9"
server_key_iv: "f6d6dde24f902dd69d53ce8d5f675761"
server_key_version: "encryptionkey@0"
I20260812 18:58:46.835644 30455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "4e6fa1c552d441ba8f03f88662a4ab11"
format_stamp: "Formatted at 2026-08-12 18:58:46 on dist-test-slave-6nmv"
server_key: "36456eb26519a0aba2da177738aa7cc9"
server_key_iv: "f6d6dde24f902dd69d53ce8d5f675761"
server_key_version: "encryptionkey@0"
I20260812 18:58:46.839226 30455 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 18:58:46.841435 30474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:46.842555 30455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:58:46.842686 30455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "4e6fa1c552d441ba8f03f88662a4ab11"
format_stamp: "Formatted at 2026-08-12 18:58:46 on dist-test-slave-6nmv"
server_key: "36456eb26519a0aba2da177738aa7cc9"
server_key_iv: "f6d6dde24f902dd69d53ce8d5f675761"
server_key_version: "encryptionkey@0"
I20260812 18:58:46.842793 30455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:46.851388 30455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:46.854086 30455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:46.854287 30455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:46.854808 30455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:58:46.855528 30455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:46.855598 30455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:46.855661 30455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:46.855703 30455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:46.864202 30455 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:41155
I20260812 18:58:46.864223 30614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:41155 every 8 connection(s)
I20260812 18:58:46.865029 30455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:58:46.867620 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 30455
I20260812 18:58:46.867748 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:58:46.867990 30147 external_mini_cluster.cc:1428] Setting key 1c6f44984f338a8188f03d5d128056e3
Aug 12 18:58:46 dist-test-slave-6nmv krb5kdc[30156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561126, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
I20260812 18:58:46.877705 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:46.866665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:52427 (local address 127.29.112.254:33521)
0812 18:58:46.866788 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:46.866791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:46.867619 (+ 828us) server_negotiation.cc:407] Connection header received
0812 18:58:46.868493 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:46.868496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:46.868547 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:46.868637 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:46.869585 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:46.870107 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:46.871171 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:46.871372 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:46.871451 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:46.874270 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:46.874274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:46.874276 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:46.874300 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:46.875713 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:46.876208 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:46.876213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:46.876215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:46.876303 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:46.876647 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:46.876650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:46.876652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:46.876829 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:46.876934 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:46.877424 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 18:58:46.877544 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 18:58:46.878391 30617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:46.866939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33521 (local address 127.29.112.194:52427)
0812 18:58:46.867501 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:46.867534 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:46.868303 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:46.868787 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:46.868792 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:46.869099 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:46.869466 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:46.869476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:46.870216 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:46.870219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:46.871025 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:46.871047 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:46.871508 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:46.871513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:46.871732 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:46.872296 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:46.872318 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:46.874095 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:46.875846 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:46.875851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:46.875870 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:46.876109 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:46.876412 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:46.876415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:46.876417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:46.876547 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:46.876945 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:46.876953 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:46.877192 (+ 239us) client_negotiation.cc:769] Sending connection context
0812 18:58:46.877370 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 18:58:46.877634 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
I20260812 18:58:46.879594 30615 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33521
I20260812 18:58:46.879770 30615 heartbeater.cc:464] Registering TS with master...
I20260812 18:58:46.880219 30615 heartbeater.cc:511] Master 127.29.112.254:33521 requested a full tablet report, sending...
I20260812 18:58:46.881173 30214 ts_manager.cc:194] Registered new tserver with Master: 4e6fa1c552d441ba8f03f88662a4ab11 (127.29.112.194:41155)
I20260812 18:58:46.881685 30214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:52427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 18:58:46.892565 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33521
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:46.997380 30620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:46.997592 30620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:46.997628 30620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:47.001101 30620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:47.001148 30620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:47.001199 30620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:58:47.005700 30620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:47.006561 30620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:47.007501 30620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:47.016382 30629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:47.016382 30626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:47.016472 30627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:47.016578 30620 server_base.cc:1034] running on GCE node
I20260812 18:58:47.017069 30620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:47.017625 30620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:47.018806 30620 hybrid_clock.cc:648] HybridClock initialized: now 1786561127018780 us; error 37 us; skew 500 ppm
Aug 12 18:58:47 dist-test-slave-6nmv krb5kdc[30156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561127, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:47.021384 30620 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 18:58:47.022406 30620 webserver.cc:460] Webserver started at http://127.29.112.195:36525/ using document root <none> and password file <none>
I20260812 18:58:47.022918 30620 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:47.023020 30620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:47.023250 30620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:47.025275 30620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "3783f4e50b2847be9aabf69ad3ef79de"
format_stamp: "Formatted at 2026-08-12 18:58:47 on dist-test-slave-6nmv"
server_key: "5db028667f59e90efbbd48f8111c8525"
server_key_iv: "6e7ec09d218f082ea766fdb7a1627f52"
server_key_version: "encryptionkey@0"
I20260812 18:58:47.025872 30620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "3783f4e50b2847be9aabf69ad3ef79de"
format_stamp: "Formatted at 2026-08-12 18:58:47 on dist-test-slave-6nmv"
server_key: "5db028667f59e90efbbd48f8111c8525"
server_key_iv: "6e7ec09d218f082ea766fdb7a1627f52"
server_key_version: "encryptionkey@0"
I20260812 18:58:47.029531 30620 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 18:58:47.031816 30636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:47.032931 30620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:58:47.033075 30620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "3783f4e50b2847be9aabf69ad3ef79de"
format_stamp: "Formatted at 2026-08-12 18:58:47 on dist-test-slave-6nmv"
server_key: "5db028667f59e90efbbd48f8111c8525"
server_key_iv: "6e7ec09d218f082ea766fdb7a1627f52"
server_key_version: "encryptionkey@0"
I20260812 18:58:47.033186 30620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:47.051396 30620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:47.054198 30620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:47.054390 30620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:47.054935 30620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:58:47.055719 30620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:47.055794 30620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:47.055858 30620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:47.055912 30620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:47.067086 30620 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:38773
I20260812 18:58:47.067111 30773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:38773 every 8 connection(s)
I20260812 18:58:47.067934 30620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
Aug 12 18:58:47 dist-test-slave-6nmv krb5kdc[30156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561127, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:47.077998 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 30620
I20260812 18:58:47.078089 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:58:47.078342 30147 external_mini_cluster.cc:1428] Setting key 779a024c5573c324d19762d23b36af0f
I20260812 18:58:47.082084 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:47.069764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:48033 (local address 127.29.112.254:33521)
0812 18:58:47.069936 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:47.069955 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:47.070962 (+ 1007us) server_negotiation.cc:407] Connection header received
0812 18:58:47.071777 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:47.071780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:47.071825 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:47.071898 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:47.072972 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:47.073633 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:47.075214 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:47.075593 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:47.075704 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:47.078377 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:47.078381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:47.078384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:47.078448 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:47.080016 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:47.080509 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:47.080512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:47.080514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:47.080643 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:47.081045 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:47.081048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:47.081049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:47.081279 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:47.081369 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:47.081806 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 18:58:47.081972 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 18:58:47.082890 30777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:47.070204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33521 (local address 127.29.112.195:48033)
0812 18:58:47.070808 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:47.070846 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:47.071594 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:47.072051 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:47.072057 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:47.072389 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:47.072846 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:47.072859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:47.073790 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:47.073794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:47.075075 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:47.075089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:47.075769 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:47.075774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:47.076020 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:47.076680 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:47.076696 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:47.078217 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:47.080130 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:47.080136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:47.080149 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:47.080394 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:47.080749 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:47.080752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:47.080754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:47.080931 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:47.081380 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:47.081384 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:47.081597 (+ 213us) client_negotiation.cc:769] Sending connection context
0812 18:58:47.081735 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 18:58:47.082033 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":105,"threads_started":1}
I20260812 18:58:47.084091 30774 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33521
I20260812 18:58:47.084323 30774 heartbeater.cc:464] Registering TS with master...
I20260812 18:58:47.084766 30774 heartbeater.cc:511] Master 127.29.112.254:33521 requested a full tablet report, sending...
I20260812 18:58:47.085732 30214 ts_manager.cc:194] Registered new tserver with Master: 3783f4e50b2847be9aabf69ad3ef79de (127.29.112.195:38773)
I20260812 18:58:47.086290 30214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:48033
I20260812 18:58:47.092556 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 18:58:47 dist-test-slave-6nmv krb5kdc[30156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561127, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 18:58:47 dist-test-slave-6nmv krb5kdc[30156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561127, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.193@KRBTEST.COM
I20260812 18:58:47.118285 30782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:47.109720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.29.112.193:38631)
0812 18:58:47.110066 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:47.110074 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:47.110095 (+ 21us) server_negotiation.cc:407] Connection header received
0812 18:58:47.110141 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:47.110145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:47.110303 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:47.110410 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:47.110975 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:47.111691 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:47.112470 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:47.112817 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:47.112896 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:47.115253 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:47.115261 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:47.115272 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:47.115292 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:47.116697 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:47.117083 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:47.117088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:47.117089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:47.117174 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:47.117392 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:47.117394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:47.117395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:47.117612 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:47.117760 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:47.118007 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 18:58:47.118133 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":139,"threads_started":1}
W20260812 18:58:47.119201 30343 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53272
Aug 12 18:58:47 dist-test-slave-6nmv krb5kdc[30156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 18:58:47 dist-test-slave-6nmv krb5kdc[30156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.193@KRBTEST.COM
I20260812 18:58:47.144797 30782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:47.137320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53278 (local address 127.29.112.193:38631)
0812 18:58:47.137474 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:47.137478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:47.137627 (+ 149us) server_negotiation.cc:407] Connection header received
0812 18:58:47.137785 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:47.137789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:47.137899 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:47.137995 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:47.138639 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:47.139110 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:47.139838 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:47.140061 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:47.140135 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:47.142280 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:47.142283 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:47.142285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:47.142306 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:47.143517 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:47.143870 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:47.143872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:47.143873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:47.143949 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:47.144170 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:47.144172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:47.144174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:47.144316 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:47.144423 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:47.144607 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 18:58:47.144690 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 18:58:47.145879 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 30292
I20260812 18:58:47.156900 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 30455
I20260812 18:58:47.169512 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 30620
I20260812 18:58:47.175633 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 30171
2026-08-12T18:58:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2156 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:58:47 dist-test-slave-6nmv krb5kdc[30791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:58:47 dist-test-slave-6nmv krb5kdc[30791](info): set up 2 sockets
Aug 12 18:58:47 dist-test-slave-6nmv krb5kdc[30791](info): commencing operation
krb5kdc: starting...
W20260812 18:58:48.286113 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:58:48 dist-test-slave-6nmv krb5kdc[30791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:58:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:58:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:58:48.404736 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:41883
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:41883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:48.510017 30804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:48.510237 30804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:48.510274 30804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:48.514132 30804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:58:48.514185 30804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:48.514207 30804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:48.514226 30804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:58:48.514286 30804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:58:48.519071 30804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:41883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:41883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:48.519966 30804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:48.520928 30804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:48.529174 30811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:48.529170 30810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:48.529168 30813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:48.529209 30804 server_base.cc:1034] running on GCE node
I20260812 18:58:48.529949 30804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:48.530860 30804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:48.532048 30804 hybrid_clock.cc:648] HybridClock initialized: now 1786561128532028 us; error 44 us; skew 500 ppm
Aug 12 18:58:48 dist-test-slave-6nmv krb5kdc[30791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561128, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:48.538148 30804 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:58:48.538962 30804 webserver.cc:460] Webserver started at http://127.29.112.254:44053/ using document root <none> and password file <none>
I20260812 18:58:48.539454 30804 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:48.539537 30804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:48.539745 30804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:48.541559 30804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "77d2ce48f25b47a8b302106225fcaa75"
format_stamp: "Formatted at 2026-08-12 18:58:48 on dist-test-slave-6nmv"
server_key: "b678ce5339a0af3daf6dcdfab199bc39"
server_key_iv: "2b90d095710111365c950b54c7aff880"
server_key_version: "encryptionkey@0"
I20260812 18:58:48.542114 30804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "77d2ce48f25b47a8b302106225fcaa75"
format_stamp: "Formatted at 2026-08-12 18:58:48 on dist-test-slave-6nmv"
server_key: "b678ce5339a0af3daf6dcdfab199bc39"
server_key_iv: "2b90d095710111365c950b54c7aff880"
server_key_version: "encryptionkey@0"
I20260812 18:58:48.545535 30804 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 18:58:48.547930 30820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:48.549006 30804 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:58:48.549150 30804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "77d2ce48f25b47a8b302106225fcaa75"
format_stamp: "Formatted at 2026-08-12 18:58:48 on dist-test-slave-6nmv"
server_key: "b678ce5339a0af3daf6dcdfab199bc39"
server_key_iv: "2b90d095710111365c950b54c7aff880"
server_key_version: "encryptionkey@0"
I20260812 18:58:48.549254 30804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:48.589686 30804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:48.592594 30804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:48.592803 30804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:48.599776 30804 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:41883
I20260812 18:58:48.599848 30889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:41883 every 8 connection(s)
I20260812 18:58:48.601013 30804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:58:48.604818 30890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:48.611111 30890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75: Bootstrap starting.
I20260812 18:58:48.611158 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 30804
I20260812 18:58:48.611250 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:58:48.611537 30147 external_mini_cluster.cc:1428] Setting key 9c52e479138a85178547e7d09bb39613
I20260812 18:58:48.613780 30890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:48.614557 30890 log.cc:824] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:48.616617 30890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75: No bootstrap required, opened a new log
I20260812 18:58:48.620316 30890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d2ce48f25b47a8b302106225fcaa75" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41883 } }
I20260812 18:58:48.620566 30890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:48.620623 30890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d2ce48f25b47a8b302106225fcaa75, State: Initialized, Role: FOLLOWER
Aug 12 18:58:48 dist-test-slave-6nmv krb5kdc[30791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:48.621043 30890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d2ce48f25b47a8b302106225fcaa75" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41883 } }
I20260812 18:58:48.621201 30890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:58:48.621277 30890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:58:48.621371 30890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:48.622488 30890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d2ce48f25b47a8b302106225fcaa75" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41883 } }
I20260812 18:58:48.622802 30890 leader_election.cc:302] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77d2ce48f25b47a8b302106225fcaa75; no voters:
I20260812 18:58:48.623124 30890 leader_election.cc:288] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:58:48.623191 30896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:48.623476 30896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [term 1 LEADER]: Becoming Leader. State: Replica: 77d2ce48f25b47a8b302106225fcaa75, State: Running, Role: LEADER
I20260812 18:58:48.623831 30896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d2ce48f25b47a8b302106225fcaa75" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41883 } }
I20260812 18:58:48.624354 30890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:58:48.624925 30896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77d2ce48f25b47a8b302106225fcaa75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d2ce48f25b47a8b302106225fcaa75" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41883 } } }
I20260812 18:58:48.625092 30896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [sys.catalog]: This master's current role is: LEADER
I20260812 18:58:48.626063 30897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77d2ce48f25b47a8b302106225fcaa75. Latest consensus state: current_term: 1 leader_uuid: "77d2ce48f25b47a8b302106225fcaa75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d2ce48f25b47a8b302106225fcaa75" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41883 } } }
I20260812 18:58:48.626154 30897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75 [sys.catalog]: This master's current role is: LEADER
I20260812 18:58:48.627280 30893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:48.612910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.29.112.254:41883)
0812 18:58:48.613345 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:48.613364 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:48.613414 (+ 50us) server_negotiation.cc:407] Connection header received
0812 18:58:48.614233 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:48.614255 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:48.614679 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:48.615024 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:48.615851 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:48.617176 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:48.618263 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:48.618779 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:48.618944 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:48.621357 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:48.621362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:48.621378 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:48.621409 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:48.623219 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:48.623754 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:48.623760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:48.623765 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:48.623934 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:48.624368 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:48.624371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:48.624373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:48.624839 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:48.625025 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:48.626364 (+ 1339us) server_negotiation.cc:299] Negotiation successful
0812 18:58:48.626657 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":134,"threads_started":1}
W20260812 18:58:48.628927 30913 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 18:58:48.629029 30913 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 18:58:48.629500 30907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:58:48.630344 30907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:58:48.635354 30907 catalog_manager.cc:1347] Generated new cluster ID: a855e5d9e90349c1a3de28871fe3854a
I20260812 18:58:48.635423 30907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:58:48.648854 30907 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:58:48.649971 30907 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:58:48.670671 30907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77d2ce48f25b47a8b302106225fcaa75: Generated new TSK 0
I20260812 18:58:48.671479 30907 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 18:58:48.702848 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41883
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:48.811976 30921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:48.812250 30921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:48.812335 30921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:48.816452 30921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:48.816543 30921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:48.816640 30921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:58:48.821789 30921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:48.823036 30921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:48.824062 30921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:48.833511 30927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:48.833527 30929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:48.833598 30926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:48.833777 30921 server_base.cc:1034] running on GCE node
I20260812 18:58:48.834169 30921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:48.834823 30921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:48.835984 30921 hybrid_clock.cc:648] HybridClock initialized: now 1786561128835973 us; error 26 us; skew 500 ppm
Aug 12 18:58:48 dist-test-slave-6nmv krb5kdc[30791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561128, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:48.838724 30921 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 18:58:48.839465 30921 webserver.cc:460] Webserver started at http://127.29.112.193:33885/ using document root <none> and password file <none>
I20260812 18:58:48.839984 30921 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:48.840039 30921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:48.840189 30921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:48.842096 30921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "d85e60ded30247c6904a173a6b97c781"
format_stamp: "Formatted at 2026-08-12 18:58:48 on dist-test-slave-6nmv"
server_key: "9c639568987d38192a6b9687fff75374"
server_key_iv: "8db2247191db42481e5e37a7ec3818dd"
server_key_version: "encryptionkey@0"
I20260812 18:58:48.842600 30921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "d85e60ded30247c6904a173a6b97c781"
format_stamp: "Formatted at 2026-08-12 18:58:48 on dist-test-slave-6nmv"
server_key: "9c639568987d38192a6b9687fff75374"
server_key_iv: "8db2247191db42481e5e37a7ec3818dd"
server_key_version: "encryptionkey@0"
I20260812 18:58:48.846236 30921 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260812 18:58:48.848398 30938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:48.849467 30921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 18:58:48.849591 30921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "d85e60ded30247c6904a173a6b97c781"
format_stamp: "Formatted at 2026-08-12 18:58:48 on dist-test-slave-6nmv"
server_key: "9c639568987d38192a6b9687fff75374"
server_key_iv: "8db2247191db42481e5e37a7ec3818dd"
server_key_version: "encryptionkey@0"
I20260812 18:58:48.849689 30921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:48.867879 30921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:48.870452 30921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:48.870643 30921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:48.871162 30921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:58:48.872057 30921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:48.872114 30921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:48.872156 30921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:48.872182 30921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:48.880652 30921 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:38879
I20260812 18:58:48.880681 31075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:38879 every 8 connection(s)
I20260812 18:58:48.881479 30921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:58:48.889240 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 30921
I20260812 18:58:48.889364 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:58:48.889662 30147 external_mini_cluster.cc:1428] Setting key b649bf42b25712330041bcadd5dd795e
Aug 12 18:58:48 dist-test-slave-6nmv krb5kdc[30791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561128, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:48.894982 30893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:48.883118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:51353 (local address 127.29.112.254:41883)
0812 18:58:48.883275 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:48.883280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:48.884065 (+ 785us) server_negotiation.cc:407] Connection header received
0812 18:58:48.885679 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:48.885683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:48.885736 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:48.885849 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:48.886910 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:48.887397 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:48.888635 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:48.888862 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:48.888978 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:48.891456 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:48.891459 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:48.891462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:48.891490 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:48.892935 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:48.893475 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:48.893480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:48.893482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:48.893565 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:48.893919 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:48.893924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:48.893925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:48.894109 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:48.894215 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:48.894710 (+ 495us) server_negotiation.cc:299] Negotiation successful
0812 18:58:48.894829 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 18:58:48.895830 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:48.883504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:41883 (local address 127.29.112.193:51353)
0812 18:58:48.883916 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:48.883955 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:48.885462 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:48.886025 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:48.886031 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:48.886366 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:48.886774 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:48.886786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:48.887545 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:48.887549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:48.888500 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:48.888510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:48.889020 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:48.889024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:48.889176 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:48.889712 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:48.889726 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:48.891296 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:48.893093 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:48.893098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:48.893111 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:48.893369 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:48.893659 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:48.893663 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:48.893665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:48.893801 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:48.894225 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:48.894233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:48.894468 (+ 235us) client_negotiation.cc:769] Sending connection context
0812 18:58:48.894655 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 18:58:48.894934 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":107,"threads_started":1}
I20260812 18:58:48.897037 31076 heartbeater.cc:347] Connected to a master server at 127.29.112.254:41883
I20260812 18:58:48.897324 31076 heartbeater.cc:464] Registering TS with master...
I20260812 18:58:48.897908 31076 heartbeater.cc:511] Master 127.29.112.254:41883 requested a full tablet report, sending...
I20260812 18:58:48.899585 30841 ts_manager.cc:194] Registered new tserver with Master: d85e60ded30247c6904a173a6b97c781 (127.29.112.193:38879)
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 18:58:48.900851 30841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:51353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 18:58:48.916566 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41883
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:49.021524 31082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:49.021783 31082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:49.021838 31082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:49.025668 31082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:49.025717 31082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:49.025767 31082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:58:49.030471 31082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:49.031306 31082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:49.032174 31082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:49.040895 31088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:49.040920 31089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:49.040901 31091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:49.041215 31082 server_base.cc:1034] running on GCE node
I20260812 18:58:49.041579 31082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:49.042200 31082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:49.043366 31082 hybrid_clock.cc:648] HybridClock initialized: now 1786561129043349 us; error 26 us; skew 500 ppm
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[30791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:49.045962 31082 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 18:58:49.046669 31082 webserver.cc:460] Webserver started at http://127.29.112.194:44681/ using document root <none> and password file <none>
I20260812 18:58:49.047171 31082 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:49.047226 31082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:49.047369 31082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:49.049288 31082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "636df8f947154121a945b0a7a43d0d32"
format_stamp: "Formatted at 2026-08-12 18:58:49 on dist-test-slave-6nmv"
server_key: "accde7017b7a370d573922e9d9eff1d6"
server_key_iv: "3bc5500400084734be76d76d1b09b4b1"
server_key_version: "encryptionkey@0"
I20260812 18:58:49.049768 31082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "636df8f947154121a945b0a7a43d0d32"
format_stamp: "Formatted at 2026-08-12 18:58:49 on dist-test-slave-6nmv"
server_key: "accde7017b7a370d573922e9d9eff1d6"
server_key_iv: "3bc5500400084734be76d76d1b09b4b1"
server_key_version: "encryptionkey@0"
I20260812 18:58:49.053612 31082 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 18:58:49.055975 31098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:49.057288 31082 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 18:58:49.057403 31082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "636df8f947154121a945b0a7a43d0d32"
format_stamp: "Formatted at 2026-08-12 18:58:49 on dist-test-slave-6nmv"
server_key: "accde7017b7a370d573922e9d9eff1d6"
server_key_iv: "3bc5500400084734be76d76d1b09b4b1"
server_key_version: "encryptionkey@0"
I20260812 18:58:49.057512 31082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:49.091077 31082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:49.094205 31082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:49.094414 31082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:49.095047 31082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:58:49.095886 31082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:49.095961 31082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:49.096037 31082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:49.096086 31082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:49.105046 31082 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:32833
I20260812 18:58:49.105064 31244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:32833 every 8 connection(s)
I20260812 18:58:49.105978 31082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:58:49.112624 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 31082
I20260812 18:58:49.112766 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:58:49.113055 30147 external_mini_cluster.cc:1428] Setting key 86e7cd2b51501d277d1308c3f3c5dbfc
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[30791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:49.119979 30893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.107749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:58345 (local address 127.29.112.254:41883)
0812 18:58:49.107881 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:49.107885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:49.108682 (+ 797us) server_negotiation.cc:407] Connection header received
0812 18:58:49.110040 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:49.110043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:49.110103 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:49.110201 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:49.111316 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.111867 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.113097 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.113375 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.113488 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.116198 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:49.116204 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:49.116210 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:49.116243 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:49.117764 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.118259 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.118265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.118269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.118364 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.118674 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.118679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.118684 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.118954 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:49.119079 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:49.119608 (+ 529us) server_negotiation.cc:299] Negotiation successful
0812 18:58:49.119741 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 18:58:49.120555 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.108102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:41883 (local address 127.29.112.194:58345)
0812 18:58:49.108535 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:49.108578 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:49.109775 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:49.110377 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:49.110383 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:49.110732 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:49.111177 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.111190 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.111991 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.111995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.112973 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.112982 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.113670 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.113674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.113897 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.114559 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:49.114575 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:49.116041 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:49.117898 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:49.117902 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:49.117913 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:49.118159 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:49.118458 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:49.118460 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:49.118462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:49.118583 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:49.119062 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:49.119089 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:49.119360 (+ 271us) client_negotiation.cc:769] Sending connection context
0812 18:58:49.119518 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 18:58:49.119758 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":115,"threads_started":1}
I20260812 18:58:49.121831 31245 heartbeater.cc:347] Connected to a master server at 127.29.112.254:41883
I20260812 18:58:49.122093 31245 heartbeater.cc:464] Registering TS with master...
I20260812 18:58:49.122562 31245 heartbeater.cc:511] Master 127.29.112.254:41883 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
I20260812 18:58:49.123594 30841 ts_manager.cc:194] Registered new tserver with Master: 636df8f947154121a945b0a7a43d0d32 (127.29.112.194:32833)
I20260812 18:58:49.124183 30841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:58345
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 18:58:49.140885 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41883
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:49.245301 31250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:49.245589 31250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:49.245661 31250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:49.249616 31250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:49.249702 31250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:49.249799 31250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:58:49.255247 31250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:49.256242 31250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:49.257284 31250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:49.266884 31259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:49.266886 31257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:49.266891 31256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:49.267234 31250 server_base.cc:1034] running on GCE node
I20260812 18:58:49.267692 31250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:49.268292 31250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:49.269487 31250 hybrid_clock.cc:648] HybridClock initialized: now 1786561129269479 us; error 40 us; skew 500 ppm
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[30791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:49.272318 31250 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 18:58:49.273123 31250 webserver.cc:460] Webserver started at http://127.29.112.195:38337/ using document root <none> and password file <none>
I20260812 18:58:49.273649 31250 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:49.273705 31250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:49.273931 31250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:49.275974 31250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "b364f63dae834870a2a8ccc8447fa947"
format_stamp: "Formatted at 2026-08-12 18:58:49 on dist-test-slave-6nmv"
server_key: "f80e68c64207ded6b47272f3ace34cd5"
server_key_iv: "eeda07399dc9f9468ef3138caace174d"
server_key_version: "encryptionkey@0"
I20260812 18:58:49.276477 31250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "b364f63dae834870a2a8ccc8447fa947"
format_stamp: "Formatted at 2026-08-12 18:58:49 on dist-test-slave-6nmv"
server_key: "f80e68c64207ded6b47272f3ace34cd5"
server_key_iv: "eeda07399dc9f9468ef3138caace174d"
server_key_version: "encryptionkey@0"
I20260812 18:58:49.279959 31250 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 18:58:49.282258 31268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:49.283439 31250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:58:49.283566 31250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "b364f63dae834870a2a8ccc8447fa947"
format_stamp: "Formatted at 2026-08-12 18:58:49 on dist-test-slave-6nmv"
server_key: "f80e68c64207ded6b47272f3ace34cd5"
server_key_iv: "eeda07399dc9f9468ef3138caace174d"
server_key_version: "encryptionkey@0"
I20260812 18:58:49.283666 31250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:49.299393 31250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:49.302253 31250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:49.302446 31250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:49.302954 31250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:58:49.303709 31250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:49.303754 31250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:49.303817 31250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:49.303850 31250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:49.311800 31250 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:36927
I20260812 18:58:49.311892 31407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:36927 every 8 connection(s)
I20260812 18:58:49.312640 31250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:58:49.316425 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 31250
I20260812 18:58:49.316547 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:58:49.316780 30147 external_mini_cluster.cc:1428] Setting key d22442ec682df4fc9e5858d986c966ff
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[30791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:49.324779 30893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.314399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:35049 (local address 127.29.112.254:41883)
0812 18:58:49.314528 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:49.314532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:49.315269 (+ 737us) server_negotiation.cc:407] Connection header received
0812 18:58:49.316111 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:49.316113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:49.316170 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:49.316267 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:49.317469 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.318027 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.319387 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.319775 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.319881 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.321902 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:49.321904 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:49.321906 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:49.321928 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:49.323071 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.323528 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.323531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.323532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.323604 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.323899 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.323902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.323903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.324064 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:49.324139 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:49.324598 (+ 459us) server_negotiation.cc:299] Negotiation successful
0812 18:58:49.324680 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 18:58:49.325521 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.314731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:41883 (local address 127.29.112.195:35049)
0812 18:58:49.315140 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:49.315176 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:49.315914 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:49.316413 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:49.316419 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:49.316818 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:49.317338 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.317353 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.318141 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.318145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.319260 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.319276 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.319944 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.319947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.320084 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.320484 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:49.320497 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:49.321734 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:49.323171 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:49.323175 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:49.323183 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:49.323440 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:49.323681 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:49.323683 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:49.323684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:49.323801 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:49.324176 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:49.324180 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:49.324372 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 18:58:49.324551 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 18:58:49.324756 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":124,"threads_started":1}
I20260812 18:58:49.326653 31410 heartbeater.cc:347] Connected to a master server at 127.29.112.254:41883
I20260812 18:58:49.326874 31410 heartbeater.cc:464] Registering TS with master...
I20260812 18:58:49.327296 31410 heartbeater.cc:511] Master 127.29.112.254:41883 requested a full tablet report, sending...
I20260812 18:58:49.328227 30841 ts_manager.cc:194] Registered new tserver with Master: b364f63dae834870a2a8ccc8447fa947 (127.29.112.195:36927)
I20260812 18:58:49.328732 30841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:35049
I20260812 18:58:49.330225 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 18:58:49.339340 30893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.333271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58042 (local address 127.29.112.254:41883)
0812 18:58:49.333358 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:49.333361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:49.333582 (+ 221us) server_negotiation.cc:407] Connection header received
0812 18:58:49.333672 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:49.333674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:49.333703 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:49.333767 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:49.334453 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.334878 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.335568 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.335741 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.335806 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.336915 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:49.336917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:49.336919 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:49.336939 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:49.338169 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.338516 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.338518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.338519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.338584 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.338782 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.338784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.338785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.338957 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:49.339052 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:49.339182 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 18:58:49.339254 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 18:58:49.343753 30841 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:58:49.346683 30841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:58:49.361395 31428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.356319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45636 (local address 127.29.112.194:32833)
0812 18:58:49.356815 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:49.356824 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:49.356863 (+ 39us) server_negotiation.cc:407] Connection header received
0812 18:58:49.356925 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:49.356929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:49.357085 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:49.357636 (+ 551us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:49.357782 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.358853 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.360050 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.360840 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.360959 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.361071 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:49.361183 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 18:58:49.361264 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":131,"threads_started":1}
I20260812 18:58:49.361397 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.355949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:32833 (local address 127.0.0.1:45636)
0812 18:58:49.356445 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:49.356456 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:49.356527 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:49.357214 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:49.357217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:49.357252 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:49.357398 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.357404 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.359000 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.359004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.359922 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.359931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.360944 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.360947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.361017 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.361042 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 18:58:49.361153 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 18:58:49.361256 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":73,"threads_started":1}
I20260812 18:58:49.361536 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.355343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:36927 (local address 127.0.0.1:41240)
0812 18:58:49.356102 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:49.356135 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:49.356252 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:49.356901 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:49.356906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:49.356927 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:49.357067 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.357074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.358563 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.358567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.360131 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.360142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.361170 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.361173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.361245 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.361262 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 18:58:49.361328 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 18:58:49.361381 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":141,"spinlock_wait_cycles":5888,"thread_start_us":71,"threads_started":1}
I20260812 18:58:49.362295 31425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.355483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41240 (local address 127.29.112.195:36927)
0812 18:58:49.356467 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:49.356481 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:49.356502 (+ 21us) server_negotiation.cc:407] Connection header received
0812 18:58:49.356564 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:49.356569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:49.356753 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:49.356887 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:49.357198 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.358401 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.360294 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.361061 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.361173 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.361272 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:49.362047 (+ 775us) server_negotiation.cc:299] Negotiation successful
0812 18:58:49.362164 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":900,"thread_start_us":155,"threads_started":1}
I20260812 18:58:49.363044 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.355526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:38879 (local address 127.0.0.1:36336)
0812 18:58:49.356088 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:49.356135 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:49.356252 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:49.357127 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:49.357130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:49.357148 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:49.357419 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.357428 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.358568 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.358571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.360131 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.360142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.362489 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.362492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.362669 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.362695 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 18:58:49.362791 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 18:58:49.362891 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":161,"spinlock_wait_cycles":1664,"thread_start_us":97,"threads_started":1}
I20260812 18:58:49.363636 31427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.355968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36336 (local address 127.29.112.193:38879)
0812 18:58:49.356729 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:49.356743 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:49.356764 (+ 21us) server_negotiation.cc:407] Connection header received
0812 18:58:49.356831 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:49.356836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:49.356986 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:49.357146 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:49.357639 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.358418 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.361369 (+ 2951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.362168 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.362281 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.362415 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:49.363386 (+ 971us) server_negotiation.cc:299] Negotiation successful
0812 18:58:49.363484 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":99,"threads_started":1}
I20260812 18:58:49.364707 31156 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab5545cf1f04535abd5acf1a4c9b944 (DEFAULT_TABLE table=test-table [id=a58535700c63411d8722bbb6f91ba8c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:49.364761 31327 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab5545cf1f04535abd5acf1a4c9b944 (DEFAULT_TABLE table=test-table [id=a58535700c63411d8722bbb6f91ba8c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:49.365756 31156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab5545cf1f04535abd5acf1a4c9b944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:49.365823 31327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab5545cf1f04535abd5acf1a4c9b944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:49.366053 30993 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab5545cf1f04535abd5acf1a4c9b944 (DEFAULT_TABLE table=test-table [id=a58535700c63411d8722bbb6f91ba8c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:49.366873 30993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab5545cf1f04535abd5acf1a4c9b944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:49.371994 31429 tablet_bootstrap.cc:492] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32: Bootstrap starting.
I20260812 18:58:49.373018 31431 tablet_bootstrap.cc:492] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781: Bootstrap starting.
I20260812 18:58:49.373251 31430 tablet_bootstrap.cc:492] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947: Bootstrap starting.
I20260812 18:58:49.374784 31431 tablet_bootstrap.cc:654] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:49.375418 31429 tablet_bootstrap.cc:654] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:49.375418 31430 tablet_bootstrap.cc:654] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:49.375861 31431 log.cc:824] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:49.376281 31429 log.cc:824] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:49.376487 31430 log.cc:824] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:49.377696 31431 tablet_bootstrap.cc:492] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781: No bootstrap required, opened a new log
I20260812 18:58:49.377882 31431 ts_tablet_manager.cc:1397] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260812 18:58:49.378459 31430 tablet_bootstrap.cc:492] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947: No bootstrap required, opened a new log
I20260812 18:58:49.378652 31430 ts_tablet_manager.cc:1397] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 18:58:49.380084 31429 tablet_bootstrap.cc:492] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32: No bootstrap required, opened a new log
I20260812 18:58:49.380297 31429 ts_tablet_manager.cc:1397] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 18:58:49.381712 31431 raft_consensus.cc:348] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b364f63dae834870a2a8ccc8447fa947" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36927 } } peers { permanent_uuid: "d85e60ded30247c6904a173a6b97c781" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38879 } } peers { permanent_uuid: "636df8f947154121a945b0a7a43d0d32" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32833 } }
I20260812 18:58:49.381989 31431 raft_consensus.cc:374] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:49.382050 31431 raft_consensus.cc:729] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d85e60ded30247c6904a173a6b97c781, State: Initialized, Role: FOLLOWER
I20260812 18:58:49.382495 31431 consensus_queue.cc:260] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b364f63dae834870a2a8ccc8447fa947" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36927 } } peers { permanent_uuid: "d85e60ded30247c6904a173a6b97c781" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38879 } } peers { permanent_uuid: "636df8f947154121a945b0a7a43d0d32" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32833 } }
I20260812 18:58:49.382375 31430 raft_consensus.cc:348] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b364f63dae834870a2a8ccc8447fa947" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36927 } } peers { permanent_uuid: "d85e60ded30247c6904a173a6b97c781" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38879 } } peers { permanent_uuid: "636df8f947154121a945b0a7a43d0d32" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32833 } }
I20260812 18:58:49.382639 31430 raft_consensus.cc:374] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:49.382704 31430 raft_consensus.cc:729] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b364f63dae834870a2a8ccc8447fa947, State: Initialized, Role: FOLLOWER
I20260812 18:58:49.383306 31076 heartbeater.cc:503] Master 127.29.112.254:41883 was elected leader, sending a full tablet report...
I20260812 18:58:49.383522 31431 ts_tablet_manager.cc:1428] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 18:58:49.383575 31430 consensus_queue.cc:260] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b364f63dae834870a2a8ccc8447fa947" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36927 } } peers { permanent_uuid: "d85e60ded30247c6904a173a6b97c781" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38879 } } peers { permanent_uuid: "636df8f947154121a945b0a7a43d0d32" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32833 } }
I20260812 18:58:49.383697 31429 raft_consensus.cc:348] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b364f63dae834870a2a8ccc8447fa947" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36927 } } peers { permanent_uuid: "d85e60ded30247c6904a173a6b97c781" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38879 } } peers { permanent_uuid: "636df8f947154121a945b0a7a43d0d32" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32833 } }
I20260812 18:58:49.383971 31429 raft_consensus.cc:374] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:49.384042 31429 raft_consensus.cc:729] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 636df8f947154121a945b0a7a43d0d32, State: Initialized, Role: FOLLOWER
I20260812 18:58:49.384189 31410 heartbeater.cc:503] Master 127.29.112.254:41883 was elected leader, sending a full tablet report...
I20260812 18:58:49.384490 31430 ts_tablet_manager.cc:1428] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 18:58:49.384516 31429 consensus_queue.cc:260] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b364f63dae834870a2a8ccc8447fa947" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36927 } } peers { permanent_uuid: "d85e60ded30247c6904a173a6b97c781" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38879 } } peers { permanent_uuid: "636df8f947154121a945b0a7a43d0d32" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32833 } }
I20260812 18:58:49.385278 31245 heartbeater.cc:503] Master 127.29.112.254:41883 was elected leader, sending a full tablet report...
I20260812 18:58:49.387547 31429 ts_tablet_manager.cc:1428] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 18:58:49.490130 31438 raft_consensus.cc:482] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:58:49.490355 31438 raft_consensus.cc:504] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b364f63dae834870a2a8ccc8447fa947" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36927 } } peers { permanent_uuid: "d85e60ded30247c6904a173a6b97c781" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38879 } } peers { permanent_uuid: "636df8f947154121a945b0a7a43d0d32" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32833 } }
I20260812 18:58:49.491460 31438 leader_election.cc:288] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b364f63dae834870a2a8ccc8447fa947 (127.29.112.195:36927), 636df8f947154121a945b0a7a43d0d32 (127.29.112.194:32833)
I20260812 18:58:49.495641 31425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.491815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:58237 (local address 127.29.112.195:36927)
0812 18:58:49.491946 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:49.491950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:49.492153 (+ 203us) server_negotiation.cc:407] Connection header received
0812 18:58:49.492216 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:49.492219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:49.492269 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:49.492361 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:49.492703 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.493298 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.494594 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.495041 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.495134 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.495173 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:49.495421 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 18:58:49.495490 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 18:58:49.495651 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.491717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:36927 (local address 127.29.112.193:58237)
0812 18:58:49.491964 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:49.491981 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:49.492088 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:49.492413 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:49.492416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:49.492444 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:49.492578 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.492586 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.493459 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.493462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.494460 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.494470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.495163 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.495166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.495270 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.495306 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 18:58:49.495388 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 18:58:49.495463 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":43,"thread_start_us":99,"threads_started":1}
I20260812 18:58:49.495790 31428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.492216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:46591 (local address 127.29.112.194:32833)
0812 18:58:49.492372 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:49.492378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:49.492553 (+ 175us) server_negotiation.cc:407] Connection header received
0812 18:58:49.492634 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:49.492640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:49.492708 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:49.492764 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:49.493044 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.493633 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.494751 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.495219 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.495339 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.495404 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:49.495607 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 18:58:49.495677 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 18:58:49.495976 31442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.492079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:32833 (local address 127.29.112.193:46591)
0812 18:58:49.492446 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:49.492461 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:49.492551 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:49.492800 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:49.492814 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:49.492830 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:49.492942 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.492948 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.493751 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.493754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.494628 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:49.494637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.495335 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.495337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:49.495457 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.495479 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 18:58:49.495555 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 18:58:49.495627 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":1,"thread_start_us":148,"threads_started":1}
I20260812 18:58:49.496198 31353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab5545cf1f04535abd5acf1a4c9b944" candidate_uuid: "d85e60ded30247c6904a173a6b97c781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b364f63dae834870a2a8ccc8447fa947" is_pre_election: true
I20260812 18:58:49.496491 31353 raft_consensus.cc:2454] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d85e60ded30247c6904a173a6b97c781 in term 0.
I20260812 18:58:49.496490 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab5545cf1f04535abd5acf1a4c9b944" candidate_uuid: "d85e60ded30247c6904a173a6b97c781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "636df8f947154121a945b0a7a43d0d32" is_pre_election: true
I20260812 18:58:49.496738 31181 raft_consensus.cc:2454] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d85e60ded30247c6904a173a6b97c781 in term 0.
I20260812 18:58:49.496927 30943 leader_election.cc:302] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b364f63dae834870a2a8ccc8447fa947, d85e60ded30247c6904a173a6b97c781; no voters:
I20260812 18:58:49.497259 31438 raft_consensus.cc:2781] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:58:49.497334 31438 raft_consensus.cc:482] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:58:49.497382 31438 raft_consensus.cc:3037] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:49.498500 31438 raft_consensus.cc:504] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b364f63dae834870a2a8ccc8447fa947" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36927 } } peers { permanent_uuid: "d85e60ded30247c6904a173a6b97c781" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38879 } } peers { permanent_uuid: "636df8f947154121a945b0a7a43d0d32" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32833 } }
I20260812 18:58:49.498765 31438 leader_election.cc:288] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [CANDIDATE]: Term 1 election: Requested vote from peers b364f63dae834870a2a8ccc8447fa947 (127.29.112.195:36927), 636df8f947154121a945b0a7a43d0d32 (127.29.112.194:32833)
I20260812 18:58:49.499123 31353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab5545cf1f04535abd5acf1a4c9b944" candidate_uuid: "d85e60ded30247c6904a173a6b97c781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b364f63dae834870a2a8ccc8447fa947"
I20260812 18:58:49.499203 31181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab5545cf1f04535abd5acf1a4c9b944" candidate_uuid: "d85e60ded30247c6904a173a6b97c781" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "636df8f947154121a945b0a7a43d0d32"
I20260812 18:58:49.499241 31353 raft_consensus.cc:3037] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:49.499366 31181 raft_consensus.cc:3037] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:49.500276 31353 raft_consensus.cc:2454] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d85e60ded30247c6904a173a6b97c781 in term 1.
I20260812 18:58:49.500488 31181 raft_consensus.cc:2454] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d85e60ded30247c6904a173a6b97c781 in term 1.
I20260812 18:58:49.500613 30943 leader_election.cc:302] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b364f63dae834870a2a8ccc8447fa947, d85e60ded30247c6904a173a6b97c781; no voters:
I20260812 18:58:49.500787 31438 raft_consensus.cc:2781] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:49.501021 31438 raft_consensus.cc:686] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [term 1 LEADER]: Becoming Leader. State: Replica: d85e60ded30247c6904a173a6b97c781, State: Running, Role: LEADER
I20260812 18:58:49.501322 31438 consensus_queue.cc:237] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b364f63dae834870a2a8ccc8447fa947" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36927 } } peers { permanent_uuid: "d85e60ded30247c6904a173a6b97c781" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38879 } } peers { permanent_uuid: "636df8f947154121a945b0a7a43d0d32" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32833 } }
I20260812 18:58:49.504398 30837 catalog_manager.cc:5495] T 7ab5545cf1f04535abd5acf1a4c9b944 P d85e60ded30247c6904a173a6b97c781 reported cstate change: term changed from 0 to 1, leader changed from <none> to d85e60ded30247c6904a173a6b97c781 (127.29.112.193). New cstate: current_term: 1 leader_uuid: "d85e60ded30247c6904a173a6b97c781" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b364f63dae834870a2a8ccc8447fa947" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d85e60ded30247c6904a173a6b97c781" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "636df8f947154121a945b0a7a43d0d32" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32833 } health_report { overall_health: UNKNOWN } } }
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[30791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.193@KRBTEST.COM
I20260812 18:58:49.552279 31427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.543878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36348 (local address 127.29.112.193:38879)
0812 18:58:49.544051 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:49.544055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:49.544142 (+ 87us) server_negotiation.cc:407] Connection header received
0812 18:58:49.544243 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:49.544245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:49.544284 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:49.544352 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:49.544995 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.545468 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.546384 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.546605 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.546680 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.548961 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:49.548971 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:49.548983 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:49.549005 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:49.550667 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.551024 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.551029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.551030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.551127 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.551344 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.551346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.551347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.551586 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:49.551795 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:49.551920 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 18:58:49.552039 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 18:58:49.563613 31411 tablet.cc:2369] T 7ab5545cf1f04535abd5acf1a4c9b944 P b364f63dae834870a2a8ccc8447fa947: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[30791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561129, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[30791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561129, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.193@KRBTEST.COM
I20260812 18:58:49.579207 31427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.571440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36354 (local address 127.29.112.193:38879)
0812 18:58:49.571582 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:49.571586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:49.571756 (+ 170us) server_negotiation.cc:407] Connection header received
0812 18:58:49.571906 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:49.571911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:49.571952 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:49.572039 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:49.572694 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.573199 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.574121 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.574351 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.574460 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.576574 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:49.576582 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:49.576585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:49.576607 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:49.577729 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.578117 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.578123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.578128 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.578193 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.578443 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.578446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.578447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.578587 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:49.578730 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:49.578955 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 18:58:49.579054 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[30791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[30791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.193@KRBTEST.COM
I20260812 18:58:49.604152 31427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:49.596092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36356 (local address 127.29.112.193:38879)
0812 18:58:49.596226 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:49.596230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:49.596364 (+ 134us) server_negotiation.cc:407] Connection header received
0812 18:58:49.596530 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:49.596535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:49.596574 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:49.596661 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:49.597273 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.597772 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.598709 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:49.598914 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:49.599045 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:49.601267 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:49.601270 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:49.601271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:49.601290 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:49.602609 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.603006 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.603009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.603010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.603084 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:49.603335 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:49.603341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:49.603343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:49.603521 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:49.603658 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:49.603884 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 18:58:49.603997 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 18:58:49.605546 30972 tablet_service.cc:2908] Rejecting scan request for tablet 7ab5545cf1f04535abd5acf1a4c9b944: Uninitialized: clean time has not yet been initialized
W20260812 18:58:49.606993 31246 tablet.cc:2369] T 7ab5545cf1f04535abd5acf1a4c9b944 P 636df8f947154121a945b0a7a43d0d32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:58:49.608180 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 30921
I20260812 18:58:49.621551 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 31082
I20260812 18:58:49.629567 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 31250
I20260812 18:58:49.636796 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 30804
2026-08-12T18:58:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2461 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[31458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[31458](info): set up 2 sockets
Aug 12 18:58:49 dist-test-slave-6nmv krb5kdc[31458](info): commencing operation
krb5kdc: starting...
W20260812 18:58:50.718086 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:58:50 dist-test-slave-6nmv krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:58:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:58:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:58:50.834583 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:43231
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:43231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:50.938043 31476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:50.938300 31476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:50.938366 31476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:50.941890 31476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:58:50.941978 31476 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 18:58:50.942018 31476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:50.942075 31476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:50.942109 31476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:58:50.942142 31476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:58:50.946678 31476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:43231
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:43231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:50.947486 31476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:50.948376 31476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:50.956215 31483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:50.956271 31486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:50.956225 31484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:50.956497 31476 server_base.cc:1034] running on GCE node
I20260812 18:58:50.957042 31476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:50.958086 31476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:50.959254 31476 hybrid_clock.cc:648] HybridClock initialized: now 1786561130959236 us; error 27 us; skew 500 ppm
Aug 12 18:58:50 dist-test-slave-6nmv krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561130, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:50.961881 31476 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:58:50.962684 31476 webserver.cc:460] Webserver started at http://127.29.112.254:38739/ using document root <none> and password file <none>
I20260812 18:58:50.963214 31476 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:50.963303 31476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:50.963531 31476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:50.965411 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "487869a8c1d74f89ad5ab6e70545a589"
format_stamp: "Formatted at 2026-08-12 18:58:50 on dist-test-slave-6nmv"
server_key: "6bb4e106de5fccfe3f19bb0bd8b9b818"
server_key_iv: "6751303ca5a09a15211108df2c940ed0"
server_key_version: "encryptionkey@0"
I20260812 18:58:50.965994 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "487869a8c1d74f89ad5ab6e70545a589"
format_stamp: "Formatted at 2026-08-12 18:58:50 on dist-test-slave-6nmv"
server_key: "6bb4e106de5fccfe3f19bb0bd8b9b818"
server_key_iv: "6751303ca5a09a15211108df2c940ed0"
server_key_version: "encryptionkey@0"
I20260812 18:58:50.969561 31476 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 18:58:50.971846 31495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:50.973088 31476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 18:58:50.973251 31476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "487869a8c1d74f89ad5ab6e70545a589"
format_stamp: "Formatted at 2026-08-12 18:58:50 on dist-test-slave-6nmv"
server_key: "6bb4e106de5fccfe3f19bb0bd8b9b818"
server_key_iv: "6751303ca5a09a15211108df2c940ed0"
server_key_version: "encryptionkey@0"
I20260812 18:58:50.973373 31476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:51.001137 31476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:51.004745 31476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:51.004966 31476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:51.013224 31476 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:43231
I20260812 18:58:51.013234 31571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:43231 every 8 connection(s)
I20260812 18:58:51.014218 31476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:58:51.016845 31572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:51.020414 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 31476
I20260812 18:58:51.020571 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:58:51.020908 30147 external_mini_cluster.cc:1428] Setting key 419ecb2cf475e6d415339121f2939232
I20260812 18:58:51.022410 31572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589: Bootstrap starting.
I20260812 18:58:51.024753 31572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:51.025558 31572 log.cc:824] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:51.027621 31572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589: No bootstrap required, opened a new log
Aug 12 18:58:51 dist-test-slave-6nmv krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:51.031682 31572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487869a8c1d74f89ad5ab6e70545a589" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43231 } }
I20260812 18:58:51.031955 31572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:51.032047 31572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 487869a8c1d74f89ad5ab6e70545a589, State: Initialized, Role: FOLLOWER
I20260812 18:58:51.032531 31572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487869a8c1d74f89ad5ab6e70545a589" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43231 } }
I20260812 18:58:51.032683 31572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:58:51.032752 31572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:58:51.032864 31572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:51.033854 31572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487869a8c1d74f89ad5ab6e70545a589" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43231 } }
I20260812 18:58:51.034205 31572 leader_election.cc:302] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 487869a8c1d74f89ad5ab6e70545a589; no voters:
I20260812 18:58:51.034520 31572 leader_election.cc:288] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:58:51.034619 31579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:51.034890 31579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [term 1 LEADER]: Becoming Leader. State: Replica: 487869a8c1d74f89ad5ab6e70545a589, State: Running, Role: LEADER
I20260812 18:58:51.035274 31579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487869a8c1d74f89ad5ab6e70545a589" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43231 } }
I20260812 18:58:51.035698 31572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:58:51.036331 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.022341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40294 (local address 127.29.112.254:43231)
0812 18:58:51.022794 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.022811 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.022849 (+ 38us) server_negotiation.cc:407] Connection header received
0812 18:58:51.023536 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.023570 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.023887 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.024167 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:51.025143 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.026527 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.027583 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.028071 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.028181 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.030882 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:51.030890 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:51.030903 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:51.030931 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:51.033056 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.033493 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.033500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.033505 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.033679 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.034045 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.034048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.034049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.034523 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:51.034743 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.035186 (+ 443us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.035449 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":176,"threads_started":1}
I20260812 18:58:51.036439 31581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 487869a8c1d74f89ad5ab6e70545a589. Latest consensus state: current_term: 1 leader_uuid: "487869a8c1d74f89ad5ab6e70545a589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487869a8c1d74f89ad5ab6e70545a589" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43231 } } }
I20260812 18:58:51.036553 31581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [sys.catalog]: This master's current role is: LEADER
I20260812 18:58:51.037040 31580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "487869a8c1d74f89ad5ab6e70545a589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487869a8c1d74f89ad5ab6e70545a589" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43231 } } }
I20260812 18:58:51.037190 31580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589 [sys.catalog]: This master's current role is: LEADER
W20260812 18:58:51.040125 31595 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 18:58:51.040207 31595 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 18:58:51.040279 31587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:58:51.040895 31587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:58:51.045872 31587 catalog_manager.cc:1347] Generated new cluster ID: 18f7a26daf974e6cbe6c678759003997
I20260812 18:58:51.045967 31587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:58:51.060839 31587 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:58:51.061977 31587 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:58:51.066521 31587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589: Generated new TSK 0
I20260812 18:58:51.067123 31587 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 18:58:51.100812 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43231
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:51.234421 31602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:51.234669 31602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:51.234714 31602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:51.238734 31602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:51.238818 31602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:51.238857 31602 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 18:58:51.238888 31602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:58:51.243474 31602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.254:43231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:51.244407 31602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:51.245345 31602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:51.254488 31609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:51.254521 31612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:51.254472 31607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:51.254550 31602 server_base.cc:1034] running on GCE node
I20260812 18:58:51.255133 31602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:51.255784 31602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:51.256982 31602 hybrid_clock.cc:648] HybridClock initialized: now 1786561131256969 us; error 34 us; skew 500 ppm
Aug 12 18:58:51 dist-test-slave-6nmv krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:51.259716 31602 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 18:58:51.260529 31602 webserver.cc:460] Webserver started at http://127.29.112.193:45767/ using document root <none> and password file <none>
I20260812 18:58:51.261066 31602 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:51.261173 31602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:51.261415 31602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:51.263453 31602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "d1fd088b71fa4c3493b47934a8547148"
format_stamp: "Formatted at 2026-08-12 18:58:51 on dist-test-slave-6nmv"
server_key: "566962503ddbc9d80aa5b3a7b3d9c46a"
server_key_iv: "6479385569097485c4e99dcd6a92ea11"
server_key_version: "encryptionkey@0"
I20260812 18:58:51.264040 31602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "d1fd088b71fa4c3493b47934a8547148"
format_stamp: "Formatted at 2026-08-12 18:58:51 on dist-test-slave-6nmv"
server_key: "566962503ddbc9d80aa5b3a7b3d9c46a"
server_key_iv: "6479385569097485c4e99dcd6a92ea11"
server_key_version: "encryptionkey@0"
I20260812 18:58:51.267871 31602 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 18:58:51.270279 31620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:51.271579 31602 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:58:51.271798 31602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "d1fd088b71fa4c3493b47934a8547148"
format_stamp: "Formatted at 2026-08-12 18:58:51 on dist-test-slave-6nmv"
server_key: "566962503ddbc9d80aa5b3a7b3d9c46a"
server_key_iv: "6479385569097485c4e99dcd6a92ea11"
server_key_version: "encryptionkey@0"
I20260812 18:58:51.271938 31602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:51.299499 31602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:51.302495 31602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:51.302738 31602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:51.304337 31602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:51.304427 31602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:51.304513 31602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:51.304559 31602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 18:58:51 dist-test-slave-6nmv krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:51.317946 31602 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:43373
I20260812 18:58:51.318001 31759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:43373 every 8 connection(s)
I20260812 18:58:51.319234 31602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:58:51.320905 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.306374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43231 (local address 127.29.112.193:41835)
0812 18:58:51.306839 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:51.306877 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:51.307986 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:51.308718 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:51.308730 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:51.309118 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:51.309795 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.309845 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.311136 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.311139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.312022 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.312031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.312642 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.312646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.312822 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.313476 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:51.313507 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:51.315419 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:51.317748 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.317754 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.317767 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.318190 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.318613 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.318615 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.318617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.318740 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.319247 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:51.319252 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:51.319596 (+ 344us) client_negotiation.cc:769] Sending connection context
0812 18:58:51.319766 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 18:58:51.320051 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":143,"threads_started":1}
I20260812 18:58:51.321530 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.305143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:41835 (local address 127.29.112.254:43231)
0812 18:58:51.305279 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.305283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.306998 (+ 1715us) server_negotiation.cc:407] Connection header received
0812 18:58:51.308271 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.308276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.308355 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.308494 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:51.310012 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.310984 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.312155 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.312478 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.312593 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.315601 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:51.315604 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:51.315607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:51.315634 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:51.317620 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.318376 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.318380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.318382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.318497 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.318861 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.318865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.318867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.319130 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:51.319265 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.321306 (+ 2041us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.321415 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 18:58:51.322693 31760 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43231
I20260812 18:58:51.322870 31760 heartbeater.cc:464] Registering TS with master...
I20260812 18:58:51.323503 31760 heartbeater.cc:511] Master 127.29.112.254:43231 requested a full tablet report, sending...
I20260812 18:58:51.325256 31513 ts_manager.cc:194] Registered new tserver with Master: d1fd088b71fa4c3493b47934a8547148 (127.29.112.193:43373)
I20260812 18:58:51.326992 31513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:41835
I20260812 18:58:51.328608 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 31602
I20260812 18:58:51.328728 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:58:51.329051 30147 external_mini_cluster.cc:1428] Setting key 7c43487a17f1e3f2208f998d99f3ee40
I20260812 18:58:51.332867 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.324820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43231 (local address 127.29.112.193:43629)
0812 18:58:51.325123 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:51.325135 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:51.325235 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:51.325539 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:51.325541 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:51.325688 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:51.325771 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.325776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.326989 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.326991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.327607 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.327615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.328206 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.328210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.328305 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.328844 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:51.328858 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:51.329239 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:51.331164 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.331169 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.331173 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.331401 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.331743 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.331749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.331754 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.331856 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.332227 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:51.332232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:51.332340 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 18:58:51.332451 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 18:58:51.332614 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":145,"threads_started":1}
I20260812 18:58:51.332924 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.324891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:43629 (local address 127.29.112.254:43231)
0812 18:58:51.324991 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.324995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.325200 (+ 205us) server_negotiation.cc:407] Connection header received
0812 18:58:51.325303 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.325306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.325343 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.325426 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:51.325932 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.326854 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.327904 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.328097 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.328202 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.329400 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:51.329406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:51.329408 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:51.329438 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:51.331057 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.331514 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.331516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.331517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.331620 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.331951 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.331953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.331954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.332129 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:51.332217 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.332512 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.332666 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":38}
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 18:58:51.356709 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43231
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:51.466310 31769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:51.466531 31769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:51.466603 31769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:51.470636 31769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:51.470690 31769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:51.470755 31769 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 18:58:51.470842 31769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:58:51.475939 31769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.254:43231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:51.476886 31769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:51.477797 31769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:51.487468 31777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:51.487512 31775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:51.487481 31774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:51.487658 31769 server_base.cc:1034] running on GCE node
I20260812 18:58:51.488253 31769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:51.488793 31769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:51.489969 31769 hybrid_clock.cc:648] HybridClock initialized: now 1786561131489942 us; error 45 us; skew 500 ppm
Aug 12 18:58:51 dist-test-slave-6nmv krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:51.492703 31769 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 18:58:51.493454 31769 webserver.cc:460] Webserver started at http://127.29.112.194:33315/ using document root <none> and password file <none>
I20260812 18:58:51.494077 31769 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:51.494169 31769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:51.494385 31769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:51.496358 31769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "e2fdfed0a948490089af8da4e7c352b5"
format_stamp: "Formatted at 2026-08-12 18:58:51 on dist-test-slave-6nmv"
server_key: "eaf2eff61668391201dabe091c486801"
server_key_iv: "3c40a4279d09f5d918c39a52f2be9d0a"
server_key_version: "encryptionkey@0"
I20260812 18:58:51.496938 31769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "e2fdfed0a948490089af8da4e7c352b5"
format_stamp: "Formatted at 2026-08-12 18:58:51 on dist-test-slave-6nmv"
server_key: "eaf2eff61668391201dabe091c486801"
server_key_iv: "3c40a4279d09f5d918c39a52f2be9d0a"
server_key_version: "encryptionkey@0"
I20260812 18:58:51.500554 31769 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 18:58:51.502733 31785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:51.503785 31769 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:58:51.503914 31769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "e2fdfed0a948490089af8da4e7c352b5"
format_stamp: "Formatted at 2026-08-12 18:58:51 on dist-test-slave-6nmv"
server_key: "eaf2eff61668391201dabe091c486801"
server_key_iv: "3c40a4279d09f5d918c39a52f2be9d0a"
server_key_version: "encryptionkey@0"
I20260812 18:58:51.504025 31769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:51.524454 31769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:51.527349 31769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:51.527529 31769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:51.528831 31769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:51.528911 31769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:51.528988 31769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:51.529042 31769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 18:58:51 dist-test-slave-6nmv krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:51.539825 31769 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:38289
I20260812 18:58:51.540129 31922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:38289 every 8 connection(s)
I20260812 18:58:51.540830 31769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:58:51.542322 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 31769
I20260812 18:58:51.542420 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:58:51.542685 30147 external_mini_cluster.cc:1428] Setting key c0d8c5dc3c4213382bf094233662422b
I20260812 18:58:51.543298 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.529681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:59481 (local address 127.29.112.254:43231)
0812 18:58:51.529794 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.529797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.530711 (+ 914us) server_negotiation.cc:407] Connection header received
0812 18:58:51.531586 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.531590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.531683 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.531783 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:51.533471 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.534366 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.536377 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.536722 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.536876 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.539509 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:51.539523 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:51.539527 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:51.539553 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:51.541108 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.541690 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.541692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.541694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.541766 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.542183 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.542189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.542192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.542403 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:51.542498 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.543048 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.543154 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 18:58:51.544202 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.530150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43231 (local address 127.29.112.194:59481)
0812 18:58:51.530578 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:51.530614 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:51.531338 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:51.532111 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:51.532122 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:51.532658 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:51.533299 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.533320 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.534534 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.534538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.536235 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.536246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.536940 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.536944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.537172 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.537799 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:51.537869 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:51.539338 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:51.541233 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.541239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.541251 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.541592 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.541911 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.541914 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.541916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.542077 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.542527 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:51.542532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:51.542779 (+ 247us) client_negotiation.cc:769] Sending connection context
0812 18:58:51.542977 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 18:58:51.543285 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":123,"threads_started":1}
I20260812 18:58:51.545502 31923 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43231
I20260812 18:58:51.545631 31923 heartbeater.cc:464] Registering TS with master...
I20260812 18:58:51.546204 31923 heartbeater.cc:511] Master 127.29.112.254:43231 requested a full tablet report, sending...
I20260812 18:58:51.547358 31514 ts_manager.cc:194] Registered new tserver with Master: e2fdfed0a948490089af8da4e7c352b5 (127.29.112.194:38289)
I20260812 18:58:51.548180 31514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:59481
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
I20260812 18:58:51.555315 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.547730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:33687 (local address 127.29.112.254:43231)
0812 18:58:51.547828 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.547832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.548153 (+ 321us) server_negotiation.cc:407] Connection header received
0812 18:58:51.548262 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.548264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.548300 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.548378 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:51.548757 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.549398 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.550310 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.550679 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.550799 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.551942 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:51.551946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:51.551952 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:51.551983 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:51.553657 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.554171 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.554174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.554176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.554257 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.554521 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.554523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.554524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.554781 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:51.554857 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.555129 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.555222 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 18:58:51.555326 31931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.547824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43231 (local address 127.29.112.194:33687)
0812 18:58:51.548069 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:51.548084 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:51.548186 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:51.548391 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:51.548393 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:51.548573 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:51.548652 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.548657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.549533 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.549539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.550207 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.550217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.550797 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.550800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.550916 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.551411 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:51.551442 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:51.551794 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:51.553768 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.553770 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.553772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.554064 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.554345 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.554347 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.554349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.554436 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.554899 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:51.554901 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:51.554981 (+ 80us) client_negotiation.cc:769] Sending connection context
0812 18:58:51.555076 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 18:58:51.555191 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 18:58:51.572422 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43231
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:51.672549 31933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:51.672766 31933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:51.672807 31933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:51.676693 31933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:51.676740 31933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:51.676797 31933 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 18:58:51.676856 31933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:58:51.681344 31933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.254:43231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:51.682184 31933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:51.683061 31933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 18:58:51.691529 31933 server_base.cc:1034] running on GCE node
W20260812 18:58:51.691491 31946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:51.691633 31943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:51.691509 31942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:51.692095 31933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:51.692708 31933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:51.693883 31933 hybrid_clock.cc:648] HybridClock initialized: now 1786561131693861 us; error 43 us; skew 500 ppm
Aug 12 18:58:51 dist-test-slave-6nmv krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:51.696417 31933 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 18:58:51.697243 31933 webserver.cc:460] Webserver started at http://127.29.112.195:36409/ using document root <none> and password file <none>
I20260812 18:58:51.697778 31933 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:51.697911 31933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:51.698136 31933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:51.700040 31933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "154072e298524a10ad785ef932eb9de4"
format_stamp: "Formatted at 2026-08-12 18:58:51 on dist-test-slave-6nmv"
server_key: "6ed8b96c76ea24ea9d20dc81248519cd"
server_key_iv: "3466c6bd18f8a31151e9a97756d14bda"
server_key_version: "encryptionkey@0"
I20260812 18:58:51.700613 31933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "154072e298524a10ad785ef932eb9de4"
format_stamp: "Formatted at 2026-08-12 18:58:51 on dist-test-slave-6nmv"
server_key: "6ed8b96c76ea24ea9d20dc81248519cd"
server_key_iv: "3466c6bd18f8a31151e9a97756d14bda"
server_key_version: "encryptionkey@0"
I20260812 18:58:51.704206 31933 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 18:58:51.706588 31955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:51.707737 31933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:58:51.707871 31933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "154072e298524a10ad785ef932eb9de4"
format_stamp: "Formatted at 2026-08-12 18:58:51 on dist-test-slave-6nmv"
server_key: "6ed8b96c76ea24ea9d20dc81248519cd"
server_key_iv: "3466c6bd18f8a31151e9a97756d14bda"
server_key_version: "encryptionkey@0"
I20260812 18:58:51.707991 31933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:51.731454 31933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:51.734196 31933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:51.734396 31933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:51.735874 31933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:51.735937 31933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:51.735999 31933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:51.736042 31933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 18:58:51 dist-test-slave-6nmv krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:51.746456 31933 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:42851
I20260812 18:58:51.746598 32091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:42851 every 8 connection(s)
I20260812 18:58:51.747434 31933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:58:51.748461 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 31933
I20260812 18:58:51.748553 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:58:51.748800 30147 external_mini_cluster.cc:1428] Setting key 44f293465cc00ec0b70af6ab0eaf33e7
I20260812 18:58:51.749635 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.736513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:49625 (local address 127.29.112.254:43231)
0812 18:58:51.736656 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.736660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.738098 (+ 1438us) server_negotiation.cc:407] Connection header received
0812 18:58:51.739483 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.739488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.739550 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.739644 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:51.741119 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.741686 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.742885 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.743107 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.743204 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.745912 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:51.745915 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:51.745917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:51.745939 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:51.747567 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.748131 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.748136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.748139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.748232 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.748609 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.748614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.748617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.748791 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:51.748890 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.749371 (+ 481us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.749478 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 18:58:51.750430 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.737429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43231 (local address 127.29.112.195:49625)
0812 18:58:51.737927 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:51.737982 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:51.739233 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:51.739807 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:51.739815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:51.740293 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:51.740957 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.740978 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.741835 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.741841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.742767 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.742776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.743247 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.743251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.743476 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.744068 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:51.744083 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:51.745737 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:51.747659 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.747663 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.747675 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.748011 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.748331 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.748334 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.748336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.748478 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.748944 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:51.748948 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:51.749135 (+ 187us) client_negotiation.cc:769] Sending connection context
0812 18:58:51.749311 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 18:58:51.749572 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":133,"threads_started":1}
I20260812 18:58:51.751482 32092 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43231
I20260812 18:58:51.751710 32092 heartbeater.cc:464] Registering TS with master...
I20260812 18:58:51.752164 32092 heartbeater.cc:511] Master 127.29.112.254:43231 requested a full tablet report, sending...
I20260812 18:58:51.753166 31513 ts_manager.cc:194] Registered new tserver with Master: 154072e298524a10ad785ef932eb9de4 (127.29.112.195:42851)
I20260812 18:58:51.753795 31513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:49625
I20260812 18:58:51.759415 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.753854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:35753 (local address 127.29.112.254:43231)
0812 18:58:51.753958 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.753962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.753980 (+ 18us) server_negotiation.cc:407] Connection header received
0812 18:58:51.754019 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.754022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.754064 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.754168 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:51.754574 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.755042 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.755769 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.755945 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.756044 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.756894 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:51.756897 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:51.756899 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:51.756925 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:51.757995 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.758365 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.758369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.758370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.758439 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.758689 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.758693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.758695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.758862 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:51.758960 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.759200 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.759290 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 18:58:51.759424 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.753549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43231 (local address 127.29.112.195:35753)
0812 18:58:51.753840 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:51.753854 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:51.753940 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:51.754185 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:51.754188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:51.754370 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:51.754455 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.754460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.755153 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.755156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.755651 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.755658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.756048 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.756051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.756134 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.756519 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:51.756529 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:51.756777 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:51.758104 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.758107 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.758108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.758276 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.758529 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:51.758532 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:51.758534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:51.758607 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:51.758952 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:51.758954 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:51.759041 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 18:58:51.759139 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 18:58:51.759267 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":128,"threads_started":1}
I20260812 18:58:51.762667 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 18:58:51 dist-test-slave-6nmv krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 18:58:51 dist-test-slave-6nmv krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:51.787364 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.779875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.29.112.254:43231)
0812 18:58:51.780034 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.780037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.780046 (+ 9us) server_negotiation.cc:407] Connection header received
0812 18:58:51.780071 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.780073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.780103 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.780178 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:51.780797 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.781248 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.782048 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.782248 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.782341 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.784466 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:51.784470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:51.784472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:51.784502 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:51.785703 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.786163 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.786166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.786168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.786247 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:51.786524 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:51.786529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:51.786533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:51.786697 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:51.786836 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.787096 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.787208 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 18:58:51.789872 31513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:58:51.792739 31513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:58:51.806236 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.800346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55114 (local address 127.29.112.195:42851)
0812 18:58:51.801325 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.801336 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.801356 (+ 20us) server_negotiation.cc:407] Connection header received
0812 18:58:51.801425 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.801430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.801632 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.801785 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:51.802103 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.803210 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.804974 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.805559 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.805709 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.805850 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.806017 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.806101 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":891,"thread_start_us":108,"threads_started":1}
I20260812 18:58:51.806237 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.800240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:42851 (local address 127.0.0.1:55114)
0812 18:58:51.800920 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:51.800961 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:51.801068 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:51.801777 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:51.801782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:51.801802 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:51.801968 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.801978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.803319 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.803322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.804841 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.804851 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.805683 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.805688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.805791 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.805869 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 18:58:51.805990 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 18:58:51.806101 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":56,"thread_start_us":73,"threads_started":1}
I20260812 18:58:51.806591 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.800946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45676 (local address 127.29.112.193:43373)
0812 18:58:51.801332 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.801347 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.801625 (+ 278us) server_negotiation.cc:407] Connection header received
0812 18:58:51.801694 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.801698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.801872 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.802012 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:51.802292 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.803349 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.804903 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.805621 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.805755 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.805893 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.806320 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.806420 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":81,"threads_started":1}
I20260812 18:58:51.806684 32112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.800332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:38289 (local address 127.0.0.1:33272)
0812 18:58:51.801241 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:51.801253 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:51.801402 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:51.802053 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:51.802056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:51.802071 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:51.802252 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.802258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.803723 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.803726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.805065 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.805073 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.806313 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.806316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.806388 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.806425 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 18:58:51.806493 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 18:58:51.806550 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":276,"thread_start_us":88,"threads_started":1}
I20260812 18:58:51.807051 32117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.800718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:43373 (local address 127.0.0.1:45676)
0812 18:58:51.801465 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:51.801477 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:51.801548 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:51.802005 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:51.802010 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:51.802030 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:51.802169 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.802175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.803459 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.803464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.804782 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.804803 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.805961 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.805964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.806061 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.806088 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 18:58:51.806155 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 18:58:51.806219 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":87,"thread_start_us":107,"threads_started":1}
I20260812 18:58:51.807156 32116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.800725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33272 (local address 127.29.112.194:38289)
0812 18:58:51.801100 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.801112 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.801322 (+ 210us) server_negotiation.cc:407] Connection header received
0812 18:58:51.801552 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.801557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.801731 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.801876 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:51.802410 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.803568 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.805195 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.805986 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.806125 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.806243 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.806898 (+ 655us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.807008 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":94,"threads_started":1}
I20260812 18:58:51.809438 32014 tablet_service.cc:1467] Processing CreateTablet for tablet 0931c28140a5444f93755cfac2bb76e2 (DEFAULT_TABLE table=test-table [id=5846e4811b874458b7b60f67d9e93462]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:51.809438 31846 tablet_service.cc:1467] Processing CreateTablet for tablet 0931c28140a5444f93755cfac2bb76e2 (DEFAULT_TABLE table=test-table [id=5846e4811b874458b7b60f67d9e93462]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:51.810505 32014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0931c28140a5444f93755cfac2bb76e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:51.810526 31846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0931c28140a5444f93755cfac2bb76e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:51.811376 31677 tablet_service.cc:1467] Processing CreateTablet for tablet 0931c28140a5444f93755cfac2bb76e2 (DEFAULT_TABLE table=test-table [id=5846e4811b874458b7b60f67d9e93462]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:51.812355 31677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0931c28140a5444f93755cfac2bb76e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:51.817927 32119 tablet_bootstrap.cc:492] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148: Bootstrap starting.
I20260812 18:58:51.818785 32120 tablet_bootstrap.cc:492] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: Bootstrap starting.
I20260812 18:58:51.820103 32119 tablet_bootstrap.cc:654] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:51.820851 32120 tablet_bootstrap.cc:654] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:51.821045 32119 log.cc:824] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:51.821681 32120 log.cc:824] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:51.821918 32121 tablet_bootstrap.cc:492] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: Bootstrap starting.
I20260812 18:58:51.823015 32119 tablet_bootstrap.cc:492] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148: No bootstrap required, opened a new log
I20260812 18:58:51.823199 32119 ts_tablet_manager.cc:1397] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 18:58:51.823475 32120 tablet_bootstrap.cc:492] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: No bootstrap required, opened a new log
I20260812 18:58:51.823657 32120 ts_tablet_manager.cc:1397] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 18:58:51.824157 32121 tablet_bootstrap.cc:654] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:51.824957 32121 log.cc:824] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:51.826243 32119 raft_consensus.cc:348] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } }
I20260812 18:58:51.826438 32119 raft_consensus.cc:374] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:51.826484 32119 raft_consensus.cc:729] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1fd088b71fa4c3493b47934a8547148, State: Initialized, Role: FOLLOWER
I20260812 18:58:51.826716 32121 tablet_bootstrap.cc:492] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: No bootstrap required, opened a new log
I20260812 18:58:51.826874 32121 ts_tablet_manager.cc:1397] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 18:58:51.826934 32119 consensus_queue.cc:260] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } }
I20260812 18:58:51.827176 32120 raft_consensus.cc:348] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } }
I20260812 18:58:51.827395 32120 raft_consensus.cc:374] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:51.827459 32120 raft_consensus.cc:729] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 154072e298524a10ad785ef932eb9de4, State: Initialized, Role: FOLLOWER
I20260812 18:58:51.827497 31760 heartbeater.cc:503] Master 127.29.112.254:43231 was elected leader, sending a full tablet report...
I20260812 18:58:51.827819 32119 ts_tablet_manager.cc:1428] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 18:58:51.827808 32120 consensus_queue.cc:260] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } }
I20260812 18:58:51.828343 32092 heartbeater.cc:503] Master 127.29.112.254:43231 was elected leader, sending a full tablet report...
I20260812 18:58:51.828672 32120 ts_tablet_manager.cc:1428] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 18:58:51.831982 32121 raft_consensus.cc:348] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } }
I20260812 18:58:51.832190 32121 raft_consensus.cc:374] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:51.832247 32121 raft_consensus.cc:729] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2fdfed0a948490089af8da4e7c352b5, State: Initialized, Role: FOLLOWER
I20260812 18:58:51.832567 32121 consensus_queue.cc:260] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } }
I20260812 18:58:51.833086 31923 heartbeater.cc:503] Master 127.29.112.254:43231 was elected leader, sending a full tablet report...
I20260812 18:58:51.833356 32121 ts_tablet_manager.cc:1428] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260812 18:58:51.986212 32129 raft_consensus.cc:482] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:58:51.986474 32129 raft_consensus.cc:504] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } }
I20260812 18:58:51.987681 32129 leader_election.cc:288] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 154072e298524a10ad785ef932eb9de4 (127.29.112.195:42851), d1fd088b71fa4c3493b47934a8547148 (127.29.112.193:43373)
I20260812 18:58:51.991712 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.988019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:42851 (local address 127.29.112.194:38109)
0812 18:58:51.988178 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:51.988195 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:51.988289 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:51.988622 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:51.988625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:51.988652 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:51.988789 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.988794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.989620 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.989623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.990633 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.990643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.991350 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.991352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.991445 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.991465 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 18:58:51.991526 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 18:58:51.991576 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"spinlock_wait_cycles":768}
I20260812 18:58:51.991809 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.988152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:38109 (local address 127.29.112.195:42851)
0812 18:58:51.988322 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.988329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.988377 (+ 48us) server_negotiation.cc:407] Connection header received
0812 18:58:51.988439 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.988442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.988502 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.988578 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:51.988939 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.989505 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.990770 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.991222 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.991381 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.991440 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.991569 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.991672 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 18:58:51.992483 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.988114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:58485 (local address 127.29.112.193:43373)
0812 18:58:51.988245 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:51.988251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:51.988664 (+ 413us) server_negotiation.cc:407] Connection header received
0812 18:58:51.988743 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:51.988745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:51.988786 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:51.988883 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:51.989479 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.990065 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.991145 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.991774 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.991926 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.991992 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:51.992319 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 18:58:51.992388 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 18:58:51.992504 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:51.988018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:43373 (local address 127.29.112.194:58485)
0812 18:58:51.988533 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:51.988543 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:51.988662 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:51.988887 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:51.988892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:51.988911 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:51.989331 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.989337 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.990169 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.990173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.991025 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:51.991032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:51.992064 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:51.992068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:51.992181 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:51.992209 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 18:58:51.992289 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 18:58:51.992349 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":135,"spinlock_wait_cycles":4352,"thread_start_us":185,"threads_started":1}
I20260812 18:58:51.992452 32039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0931c28140a5444f93755cfac2bb76e2" candidate_uuid: "e2fdfed0a948490089af8da4e7c352b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "154072e298524a10ad785ef932eb9de4" is_pre_election: true
I20260812 18:58:51.992782 32039 raft_consensus.cc:2454] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2fdfed0a948490089af8da4e7c352b5 in term 0.
I20260812 18:58:51.993279 31701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0931c28140a5444f93755cfac2bb76e2" candidate_uuid: "e2fdfed0a948490089af8da4e7c352b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1fd088b71fa4c3493b47934a8547148" is_pre_election: true
I20260812 18:58:51.993316 31790 leader_election.cc:302] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 154072e298524a10ad785ef932eb9de4, e2fdfed0a948490089af8da4e7c352b5; no voters:
I20260812 18:58:51.993552 31701 raft_consensus.cc:2454] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2fdfed0a948490089af8da4e7c352b5 in term 0.
I20260812 18:58:51.993589 32129 raft_consensus.cc:2781] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:58:51.993675 32129 raft_consensus.cc:482] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:58:51.993742 32129 raft_consensus.cc:3037] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:51.994872 32129 raft_consensus.cc:504] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } }
I20260812 18:58:51.995250 32129 leader_election.cc:288] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [CANDIDATE]: Term 1 election: Requested vote from peers 154072e298524a10ad785ef932eb9de4 (127.29.112.195:42851), d1fd088b71fa4c3493b47934a8547148 (127.29.112.193:43373)
I20260812 18:58:51.995621 32039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0931c28140a5444f93755cfac2bb76e2" candidate_uuid: "e2fdfed0a948490089af8da4e7c352b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "154072e298524a10ad785ef932eb9de4"
I20260812 18:58:51.995679 31701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0931c28140a5444f93755cfac2bb76e2" candidate_uuid: "e2fdfed0a948490089af8da4e7c352b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1fd088b71fa4c3493b47934a8547148"
I20260812 18:58:51.995733 32039 raft_consensus.cc:3037] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:51.995855 31701 raft_consensus.cc:3037] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:51.996902 32039 raft_consensus.cc:2454] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2fdfed0a948490089af8da4e7c352b5 in term 1.
I20260812 18:58:51.997068 31701 raft_consensus.cc:2454] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2fdfed0a948490089af8da4e7c352b5 in term 1.
I20260812 18:58:51.997267 31790 leader_election.cc:302] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 154072e298524a10ad785ef932eb9de4, e2fdfed0a948490089af8da4e7c352b5; no voters:
I20260812 18:58:51.997442 32129 raft_consensus.cc:2781] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:51.997671 32129 raft_consensus.cc:686] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 1 LEADER]: Becoming Leader. State: Replica: e2fdfed0a948490089af8da4e7c352b5, State: Running, Role: LEADER
I20260812 18:58:51.998031 32129 consensus_queue.cc:237] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } }
W20260812 18:58:51.998728 32093 tablet.cc:2369] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:58:52.001392 31513 catalog_manager.cc:5495] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2fdfed0a948490089af8da4e7c352b5 (127.29.112.194). New cstate: current_term: 1 leader_uuid: "e2fdfed0a948490089af8da4e7c352b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } health_report { overall_health: UNKNOWN } } }
W20260812 18:58:52.042004 31925 tablet.cc:2369] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:58:52.047143 32116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.043661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33276 (local address 127.29.112.194:38289)
0812 18:58:52.043803 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:52.043807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:52.043883 (+ 76us) server_negotiation.cc:407] Connection header received
0812 18:58:52.044071 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:52.044073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:52.044126 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:52.044210 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:52.044458 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.044990 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.045783 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.046058 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.046131 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.046392 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:52.046739 (+ 347us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:52.046815 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:52.046902 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 18:58:52.046945 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 18:58:52.052482 32039 raft_consensus.cc:1261] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4 [term 1 FOLLOWER]: Refusing update from remote peer e2fdfed0a948490089af8da4e7c352b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:52.052486 31701 raft_consensus.cc:1261] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148 [term 1 FOLLOWER]: Refusing update from remote peer e2fdfed0a948490089af8da4e7c352b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:52.053251 32132 consensus_queue.cc:1035] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:52.053449 32129 consensus_queue.cc:1035] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:52.065234 32136 mvcc.cc:205] Tried to move back new op lower bound from 7317754396879650816 to 7317754396666621952. Current Snapshot: MvccSnapshot[applied={T|T < 7317754396879650816 or (T in {7317754396879650816})}]
I20260812 18:58:52.073338 31514 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:52.073570 31514 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:52.076063 31514 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589: Sending DeleteTablet for 3 replicas of tablet 0931c28140a5444f93755cfac2bb76e2
I20260812 18:58:52.076797 32014 tablet_service.cc:1514] Processing DeleteTablet for tablet 0931c28140a5444f93755cfac2bb76e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:52 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:55114
I20260812 18:58:52.076969 31677 tablet_service.cc:1514] Processing DeleteTablet for tablet 0931c28140a5444f93755cfac2bb76e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:52 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:45676
I20260812 18:58:52.076974 31846 tablet_service.cc:1514] Processing DeleteTablet for tablet 0931c28140a5444f93755cfac2bb76e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:52 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:33272
I20260812 18:58:52.077384 32148 tablet_replica.cc:330] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: stopping tablet replica
I20260812 18:58:52.077404 32150 tablet_replica.cc:330] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148: stopping tablet replica
I20260812 18:58:52.077657 32148 raft_consensus.cc:2229] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:58:52.077924 32150 raft_consensus.cc:2229] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:52.077998 32148 raft_consensus.cc:2258] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:52.078289 32150 raft_consensus.cc:2258] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:52.078243 31514 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:58:52.078629 31514 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:58:52.079696 32150 ts_tablet_manager.cc:1905] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:52.080137 32148 ts_tablet_manager.cc:1905] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:52.082506 32150 ts_tablet_manager.cc:1918] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:58:52.082625 32150 log.cc:1196] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/wal/wals/0931c28140a5444f93755cfac2bb76e2
I20260812 18:58:52.082918 32150 ts_tablet_manager.cc:1939] T 0931c28140a5444f93755cfac2bb76e2 P d1fd088b71fa4c3493b47934a8547148: Deleting consensus metadata
I20260812 18:58:52.083431 32014 tablet_service.cc:1467] Processing CreateTablet for tablet 736115cf295d4b8993f447ac41a59989 (DEFAULT_TABLE table=test-table [id=5e99bb6555fc4d06bbe1f89e85278d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:52.083488 31677 tablet_service.cc:1467] Processing CreateTablet for tablet 736115cf295d4b8993f447ac41a59989 (DEFAULT_TABLE table=test-table [id=5e99bb6555fc4d06bbe1f89e85278d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:52.083500 31846 tablet_service.cc:1467] Processing CreateTablet for tablet 736115cf295d4b8993f447ac41a59989 (DEFAULT_TABLE table=test-table [id=5e99bb6555fc4d06bbe1f89e85278d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:52.083730 31846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736115cf295d4b8993f447ac41a59989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:52.083727 32014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736115cf295d4b8993f447ac41a59989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:52.083760 31677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736115cf295d4b8993f447ac41a59989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:52.085489 32119 tablet_bootstrap.cc:492] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148: Bootstrap starting.
I20260812 18:58:52.085906 32121 tablet_bootstrap.cc:492] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5: Bootstrap starting.
I20260812 18:58:52.086645 32119 tablet_bootstrap.cc:654] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:52.086970 32121 tablet_bootstrap.cc:654] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:52.087699 32120 tablet_bootstrap.cc:492] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4: Bootstrap starting.
I20260812 18:58:52.087857 32148 ts_tablet_manager.cc:1918] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:58:52.088001 32148 log.cc:1196] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/0931c28140a5444f93755cfac2bb76e2
I20260812 18:58:52.088361 32148 ts_tablet_manager.cc:1939] T 0931c28140a5444f93755cfac2bb76e2 P e2fdfed0a948490089af8da4e7c352b5: Deleting consensus metadata
I20260812 18:58:52.088497 32121 tablet_bootstrap.cc:492] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5: No bootstrap required, opened a new log
I20260812 18:58:52.088578 32121 ts_tablet_manager.cc:1397] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 18:58:52.088770 32120 tablet_bootstrap.cc:654] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:52.089077 32121 raft_consensus.cc:348] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.089198 32121 raft_consensus.cc:374] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:52.089238 32121 raft_consensus.cc:729] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2fdfed0a948490089af8da4e7c352b5, State: Initialized, Role: FOLLOWER
I20260812 18:58:52.089398 32121 consensus_queue.cc:260] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.089493 31496 catalog_manager.cc:4841] TS d1fd088b71fa4c3493b47934a8547148 (127.29.112.193:43373): tablet 0931c28140a5444f93755cfac2bb76e2 (table test-table [id=5846e4811b874458b7b60f67d9e93462]) successfully deleted
I20260812 18:58:52.089679 32121 ts_tablet_manager.cc:1428] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 18:58:52.090188 31498 catalog_manager.cc:4841] TS e2fdfed0a948490089af8da4e7c352b5 (127.29.112.194:38289): tablet 0931c28140a5444f93755cfac2bb76e2 (table test-table [id=5846e4811b874458b7b60f67d9e93462]) successfully deleted
I20260812 18:58:52.090533 32146 tablet_replica.cc:330] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: stopping tablet replica
I20260812 18:58:52.090576 32119 tablet_bootstrap.cc:492] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148: No bootstrap required, opened a new log
I20260812 18:58:52.090651 32119 ts_tablet_manager.cc:1397] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 18:58:52.090831 32146 raft_consensus.cc:2229] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:52.091125 32146 raft_consensus.cc:2258] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:52.091073 32119 raft_consensus.cc:348] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.091177 32119 raft_consensus.cc:374] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:52.091210 32119 raft_consensus.cc:729] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1fd088b71fa4c3493b47934a8547148, State: Initialized, Role: FOLLOWER
I20260812 18:58:52.091331 32119 consensus_queue.cc:260] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.092116 32119 ts_tablet_manager.cc:1428] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:58:52.092473 32120 tablet_bootstrap.cc:492] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4: No bootstrap required, opened a new log
I20260812 18:58:52.092536 32120 ts_tablet_manager.cc:1397] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 18:58:52.092887 32120 raft_consensus.cc:348] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.092963 32120 raft_consensus.cc:374] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:52.092985 32120 raft_consensus.cc:729] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 154072e298524a10ad785ef932eb9de4, State: Initialized, Role: FOLLOWER
I20260812 18:58:52.093125 32120 consensus_queue.cc:260] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.093400 32120 ts_tablet_manager.cc:1428] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 18:58:52.095522 32146 ts_tablet_manager.cc:1905] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:52.098325 32146 ts_tablet_manager.cc:1918] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:58:52.098443 32146 log.cc:1196] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/0931c28140a5444f93755cfac2bb76e2
I20260812 18:58:52.098726 32146 ts_tablet_manager.cc:1939] T 0931c28140a5444f93755cfac2bb76e2 P 154072e298524a10ad785ef932eb9de4: Deleting consensus metadata
I20260812 18:58:52.099702 31499 catalog_manager.cc:4841] TS 154072e298524a10ad785ef932eb9de4 (127.29.112.195:42851): tablet 0931c28140a5444f93755cfac2bb76e2 (table test-table [id=5846e4811b874458b7b60f67d9e93462]) successfully deleted
I20260812 18:58:52.142714 32126 raft_consensus.cc:482] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:58:52.142849 32126 raft_consensus.cc:504] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.143966 32126 leader_election.cc:288] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2fdfed0a948490089af8da4e7c352b5 (127.29.112.194:38289), 154072e298524a10ad785ef932eb9de4 (127.29.112.195:42851)
I20260812 18:58:52.148814 32116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.144310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:47053 (local address 127.29.112.194:38289)
0812 18:58:52.144444 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:52.144451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:52.144665 (+ 214us) server_negotiation.cc:407] Connection header received
0812 18:58:52.144922 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:52.144927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:52.144983 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:52.145047 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:52.145438 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.146177 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.147852 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.148223 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.148356 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.148413 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:52.148608 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 18:58:52.148689 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 18:58:52.148916 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.144543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:42361 (local address 127.29.112.195:42851)
0812 18:58:52.144658 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:52.144662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:52.144762 (+ 100us) server_negotiation.cc:407] Connection header received
0812 18:58:52.144844 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:52.144847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:52.144902 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:52.144991 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:52.145361 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.146165 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.147832 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.148404 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.148510 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.148554 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:52.148733 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 18:58:52.148802 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 18:58:52.149211 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.144201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:42851 (local address 127.29.112.193:42361)
0812 18:58:52.144667 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:52.144680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:52.144760 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:52.145002 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:52.145007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:52.145037 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:52.145228 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.145238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.146308 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.146311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.147691 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.147702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.148510 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.148524 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.148622 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.148640 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 18:58:52.148700 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 18:58:52.148763 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":81,"thread_start_us":112,"threads_started":1}
I20260812 18:58:52.149211 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.144178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:38289 (local address 127.29.112.193:47053)
0812 18:58:52.144577 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:52.144592 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:52.144753 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:52.145119 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:52.145122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:52.145139 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:52.145273 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.145279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.146305 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.146308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.147699 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.147703 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.148356 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.148360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.148464 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.148495 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 18:58:52.149001 (+ 506us) client_negotiation.cc:240] Negotiation successful
0812 18:58:52.149103 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":182,"thread_start_us":76,"threads_started":1}
I20260812 18:58:52.150027 32039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736115cf295d4b8993f447ac41a59989" candidate_uuid: "d1fd088b71fa4c3493b47934a8547148" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "154072e298524a10ad785ef932eb9de4" is_pre_election: true
I20260812 18:58:52.150184 32039 raft_consensus.cc:2454] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1fd088b71fa4c3493b47934a8547148 in term 0.
I20260812 18:58:52.150357 31867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736115cf295d4b8993f447ac41a59989" candidate_uuid: "d1fd088b71fa4c3493b47934a8547148" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2fdfed0a948490089af8da4e7c352b5" is_pre_election: true
I20260812 18:58:52.150564 31867 raft_consensus.cc:2454] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1fd088b71fa4c3493b47934a8547148 in term 0.
I20260812 18:58:52.150980 31623 leader_election.cc:302] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1fd088b71fa4c3493b47934a8547148, e2fdfed0a948490089af8da4e7c352b5; no voters:
I20260812 18:58:52.151366 32126 raft_consensus.cc:2781] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:58:52.151415 32126 raft_consensus.cc:482] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:58:52.151440 32126 raft_consensus.cc:3037] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:52.152573 32126 raft_consensus.cc:504] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.152949 32126 leader_election.cc:288] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [CANDIDATE]: Term 1 election: Requested vote from peers e2fdfed0a948490089af8da4e7c352b5 (127.29.112.194:38289), 154072e298524a10ad785ef932eb9de4 (127.29.112.195:42851)
I20260812 18:58:52.153302 31867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736115cf295d4b8993f447ac41a59989" candidate_uuid: "d1fd088b71fa4c3493b47934a8547148" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2fdfed0a948490089af8da4e7c352b5"
I20260812 18:58:52.153347 32039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736115cf295d4b8993f447ac41a59989" candidate_uuid: "d1fd088b71fa4c3493b47934a8547148" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "154072e298524a10ad785ef932eb9de4"
I20260812 18:58:52.153406 31867 raft_consensus.cc:3037] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:52.153448 32039 raft_consensus.cc:3037] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:52.154489 31867 raft_consensus.cc:2454] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1fd088b71fa4c3493b47934a8547148 in term 1.
I20260812 18:58:52.154709 32039 raft_consensus.cc:2454] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1fd088b71fa4c3493b47934a8547148 in term 1.
I20260812 18:58:52.155041 31623 leader_election.cc:302] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1fd088b71fa4c3493b47934a8547148, e2fdfed0a948490089af8da4e7c352b5; no voters:
I20260812 18:58:52.155287 32126 raft_consensus.cc:2781] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:52.155469 32126 raft_consensus.cc:686] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 1 LEADER]: Becoming Leader. State: Replica: d1fd088b71fa4c3493b47934a8547148, State: Running, Role: LEADER
I20260812 18:58:52.155697 32126 consensus_queue.cc:237] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.158063 31511 catalog_manager.cc:5495] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1fd088b71fa4c3493b47934a8547148 (127.29.112.193). New cstate: current_term: 1 leader_uuid: "d1fd088b71fa4c3493b47934a8547148" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } health_report { overall_health: UNKNOWN } } }
Aug 12 18:58:52 dist-test-slave-6nmv krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561130, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:52.196300 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.187526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43231 (local address 127.0.0.1:40310)
0812 18:58:52.187839 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:52.187858 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:52.187952 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:52.188222 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:52.188224 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:52.188594 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:52.188680 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.188685 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.189627 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.189629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.190155 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.190162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.190651 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.190653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.190746 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.191290 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:52.191300 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:52.192907 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:52.194861 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:52.194869 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:52.194873 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:52.195093 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:52.195404 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:52.195406 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:52.195407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:52.195480 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:52.195909 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:52.195913 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:52.195998 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 18:58:52.196078 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 18:58:52.196175 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":25,"thread_start_us":135,"threads_started":1}
I20260812 18:58:52.196476 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.187650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.29.112.254:43231)
0812 18:58:52.187806 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:52.187811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:52.187938 (+ 127us) server_negotiation.cc:407] Connection header received
0812 18:58:52.188042 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:52.188045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:52.188096 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:52.188195 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:52.188809 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.189521 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.190271 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.190550 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.190654 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.193046 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:52.193050 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:52.193052 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:52.193075 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:52.194741 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:52.195214 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:52.195217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:52.195219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:52.195310 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:52.195573 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:52.195577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:52.195578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:52.195793 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:52.195904 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:52.196168 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 18:58:52.196285 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":63}
I20260812 18:58:52.200448 31511 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:40310:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 18:58:52.201217 31511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:58:52.206040 31846 tablet_service.cc:1467] Processing CreateTablet for tablet df9ca88ba0114dada2479cbda93a78ab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e24d3406b4c548df9cc31f63a1df55db]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 18:58:52.206444 31846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df9ca88ba0114dada2479cbda93a78ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:52.206305 32014 tablet_service.cc:1467] Processing CreateTablet for tablet df9ca88ba0114dada2479cbda93a78ab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e24d3406b4c548df9cc31f63a1df55db]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 18:58:52.206665 32014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df9ca88ba0114dada2479cbda93a78ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:52.206038 31677 tablet_service.cc:1467] Processing CreateTablet for tablet df9ca88ba0114dada2479cbda93a78ab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e24d3406b4c548df9cc31f63a1df55db]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 18:58:52.208642 32121 tablet_bootstrap.cc:492] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: Bootstrap starting.
I20260812 18:58:52.208811 31677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df9ca88ba0114dada2479cbda93a78ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:52.209607 32121 tablet_bootstrap.cc:654] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:52.210791 32120 tablet_bootstrap.cc:492] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4: Bootstrap starting.
I20260812 18:58:52.211776 32120 tablet_bootstrap.cc:654] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:52.211784 32121 tablet_bootstrap.cc:492] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: No bootstrap required, opened a new log
I20260812 18:58:52.211859 32121 ts_tablet_manager.cc:1397] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 18:58:52.212198 32119 tablet_bootstrap.cc:492] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148: Bootstrap starting.
I20260812 18:58:52.212672 32121 raft_consensus.cc:348] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.212800 32121 raft_consensus.cc:374] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:52.212843 32121 raft_consensus.cc:729] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2fdfed0a948490089af8da4e7c352b5, State: Initialized, Role: FOLLOWER
I20260812 18:58:52.213193 32119 tablet_bootstrap.cc:654] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:52.213305 32120 tablet_bootstrap.cc:492] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4: No bootstrap required, opened a new log
I20260812 18:58:52.213375 32120 ts_tablet_manager.cc:1397] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 18:58:52.213536 32121 consensus_queue.cc:260] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.213892 32129 tablet_replica.cc:439] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } }
I20260812 18:58:52.213824 32120 raft_consensus.cc:348] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.214035 32129 tablet_replica.cc:442] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:52.214069 32120 raft_consensus.cc:374] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:52.214116 32120 raft_consensus.cc:729] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 154072e298524a10ad785ef932eb9de4, State: Initialized, Role: FOLLOWER
I20260812 18:58:52.214239 32120 consensus_queue.cc:260] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.214527 32120 ts_tablet_manager.cc:1428] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 18:58:52.214476 32127 tablet_replica.cc:439] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } }
I20260812 18:58:52.214622 32127 tablet_replica.cc:442] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:52.215657 32119 tablet_bootstrap.cc:492] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148: No bootstrap required, opened a new log
I20260812 18:58:52.215756 32119 ts_tablet_manager.cc:1397] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 18:58:52.216180 32119 raft_consensus.cc:348] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.216295 32119 raft_consensus.cc:374] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:52.216328 32119 raft_consensus.cc:729] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1fd088b71fa4c3493b47934a8547148, State: Initialized, Role: FOLLOWER
I20260812 18:58:52.216447 32119 consensus_queue.cc:260] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.216706 32119 ts_tablet_manager.cc:1428] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 18:58:52.216687 32126 tablet_replica.cc:439] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } }
I20260812 18:58:52.216791 32126 tablet_replica.cc:442] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:52.218214 32121 ts_tablet_manager.cc:1428] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260812 18:58:52.259228 32129 raft_consensus.cc:482] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:58:52.259344 32129 raft_consensus.cc:504] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.259716 32129 leader_election.cc:288] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1fd088b71fa4c3493b47934a8547148 (127.29.112.193:43373), 154072e298524a10ad785ef932eb9de4 (127.29.112.195:42851)
I20260812 18:58:52.260090 31701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9ca88ba0114dada2479cbda93a78ab" candidate_uuid: "e2fdfed0a948490089af8da4e7c352b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1fd088b71fa4c3493b47934a8547148" is_pre_election: true
I20260812 18:58:52.260119 32039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9ca88ba0114dada2479cbda93a78ab" candidate_uuid: "e2fdfed0a948490089af8da4e7c352b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "154072e298524a10ad785ef932eb9de4" is_pre_election: true
I20260812 18:58:52.260257 31701 raft_consensus.cc:2454] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2fdfed0a948490089af8da4e7c352b5 in term 0.
I20260812 18:58:52.260258 32039 raft_consensus.cc:2454] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2fdfed0a948490089af8da4e7c352b5 in term 0.
I20260812 18:58:52.260558 31786 leader_election.cc:302] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 154072e298524a10ad785ef932eb9de4, d1fd088b71fa4c3493b47934a8547148, e2fdfed0a948490089af8da4e7c352b5; no voters:
I20260812 18:58:52.260735 32129 raft_consensus.cc:2781] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:58:52.260818 32129 raft_consensus.cc:482] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:58:52.260854 32129 raft_consensus.cc:3037] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:52.261883 32129 raft_consensus.cc:504] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.262187 32129 leader_election.cc:288] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [CANDIDATE]: Term 1 election: Requested vote from peers d1fd088b71fa4c3493b47934a8547148 (127.29.112.193:43373), 154072e298524a10ad785ef932eb9de4 (127.29.112.195:42851)
I20260812 18:58:52.262534 31701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9ca88ba0114dada2479cbda93a78ab" candidate_uuid: "e2fdfed0a948490089af8da4e7c352b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1fd088b71fa4c3493b47934a8547148"
I20260812 18:58:52.262583 32039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9ca88ba0114dada2479cbda93a78ab" candidate_uuid: "e2fdfed0a948490089af8da4e7c352b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "154072e298524a10ad785ef932eb9de4"
I20260812 18:58:52.262665 31701 raft_consensus.cc:3037] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:52.262701 32039 raft_consensus.cc:3037] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:52.263681 31701 raft_consensus.cc:2454] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2fdfed0a948490089af8da4e7c352b5 in term 1.
I20260812 18:58:52.263728 32039 raft_consensus.cc:2454] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2fdfed0a948490089af8da4e7c352b5 in term 1.
I20260812 18:58:52.263943 31786 leader_election.cc:302] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1fd088b71fa4c3493b47934a8547148, e2fdfed0a948490089af8da4e7c352b5; no voters:
I20260812 18:58:52.264120 32129 raft_consensus.cc:2781] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:52.264236 32129 raft_consensus.cc:686] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [term 1 LEADER]: Becoming Leader. State: Replica: e2fdfed0a948490089af8da4e7c352b5, State: Running, Role: LEADER
I20260812 18:58:52.264367 32129 consensus_queue.cc:237] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } }
I20260812 18:58:52.265110 32132 tablet_replica.cc:439] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: TxnStatusTablet state changed. Reason: New leader e2fdfed0a948490089af8da4e7c352b5. Latest consensus state: current_term: 1 leader_uuid: "e2fdfed0a948490089af8da4e7c352b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } }
I20260812 18:58:52.265215 32132 tablet_replica.cc:442] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: This TxnStatusTablet replica's current role is: LEADER
I20260812 18:58:52.265529 32171 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 18:58:52.265780 32171 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 18:58:52.266168 31514 catalog_manager.cc:5495] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2fdfed0a948490089af8da4e7c352b5 (127.29.112.194). New cstate: current_term: 1 leader_uuid: "e2fdfed0a948490089af8da4e7c352b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } health_report { overall_health: UNKNOWN } } }
I20260812 18:58:52.284111 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.280827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:38289 (local address 127.0.0.1:33282)
0812 18:58:52.280978 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:52.280993 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:52.281097 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:52.281363 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:52.281365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:52.281372 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:58:52.281472 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.281476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.282325 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.282328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.282881 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.282888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.283352 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.283356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.283440 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.283481 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:58:52.283803 (+ 322us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:58:52.283820 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 18:58:52.283903 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 18:58:52.283986 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 18:58:52.284111 32116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.280908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33282 (local address 127.29.112.194:38289)
0812 18:58:52.281020 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:52.281027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:52.281041 (+ 14us) server_negotiation.cc:407] Connection header received
0812 18:58:52.281207 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:52.281211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:52.281254 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:52.281332 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:52.281572 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.282225 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.283025 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.283244 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.283350 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.283622 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:52.283718 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:52.283798 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:52.283926 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 18:58:52.283974 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 18:58:52.286651 32039 raft_consensus.cc:1261] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4 [term 1 FOLLOWER]: Refusing update from remote peer e2fdfed0a948490089af8da4e7c352b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:52.286701 31701 raft_consensus.cc:1261] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148 [term 1 FOLLOWER]: Refusing update from remote peer e2fdfed0a948490089af8da4e7c352b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:52.287060 32129 consensus_queue.cc:1035] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:52.287308 32129 consensus_queue.cc:1035] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:52.288378 32127 tablet_replica.cc:439] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4: TxnStatusTablet state changed. Reason: New leader e2fdfed0a948490089af8da4e7c352b5. Latest consensus state: current_term: 1 leader_uuid: "e2fdfed0a948490089af8da4e7c352b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } }
I20260812 18:58:52.288509 32127 tablet_replica.cc:442] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:52.289312 32126 tablet_replica.cc:439] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148: TxnStatusTablet state changed. Reason: New leader e2fdfed0a948490089af8da4e7c352b5. Latest consensus state: current_term: 1 leader_uuid: "e2fdfed0a948490089af8da4e7c352b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } }
I20260812 18:58:52.289417 32126 tablet_replica.cc:442] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:52.290735 32140 tablet_replica.cc:439] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2fdfed0a948490089af8da4e7c352b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } }
I20260812 18:58:52.290836 32140 tablet_replica.cc:442] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: This TxnStatusTablet replica's current role is: LEADER
I20260812 18:58:52.290954 32140 tablet_replica.cc:439] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2fdfed0a948490089af8da4e7c352b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } }
I20260812 18:58:52.291020 32140 tablet_replica.cc:442] T df9ca88ba0114dada2479cbda93a78ab P e2fdfed0a948490089af8da4e7c352b5: This TxnStatusTablet replica's current role is: LEADER
I20260812 18:58:52.291096 32135 mvcc.cc:205] Tried to move back new op lower bound from 7317754397842452480 to 7317754397756227584. Current Snapshot: MvccSnapshot[applied={T|T < 7317754397842452480 or (T in {7317754397842452480})}]
I20260812 18:58:52.291697 32126 tablet_replica.cc:439] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2fdfed0a948490089af8da4e7c352b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } }
I20260812 18:58:52.291787 32126 tablet_replica.cc:442] T df9ca88ba0114dada2479cbda93a78ab P d1fd088b71fa4c3493b47934a8547148: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:52.294193 32127 tablet_replica.cc:439] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2fdfed0a948490089af8da4e7c352b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fd088b71fa4c3493b47934a8547148" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 43373 } } peers { permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 } } peers { permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 } } }
I20260812 18:58:52.294298 32127 tablet_replica.cc:442] T df9ca88ba0114dada2479cbda93a78ab P 154072e298524a10ad785ef932eb9de4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:52.300071 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.296749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45678 (local address 127.29.112.193:43373)
0812 18:58:52.296842 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:52.296847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:52.296880 (+ 33us) server_negotiation.cc:407] Connection header received
0812 18:58:52.297040 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:52.297043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:52.297098 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:52.297194 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:52.297434 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.298013 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.298809 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.299024 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.299125 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.299368 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:52.299741 (+ 373us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:52.299858 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:52.299915 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 18:58:52.299960 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 18:58:52.309430 32116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.306224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:37663 (local address 127.29.112.194:38289)
0812 18:58:52.306328 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:52.306333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:52.306507 (+ 174us) server_negotiation.cc:407] Connection header received
0812 18:58:52.306645 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:52.306647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:52.306682 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:52.306755 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:52.307024 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.307573 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.308538 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.308734 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.308827 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.309089 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:52.309142 (+ 53us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:52.309212 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:52.309288 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 18:58:52.309328 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 18:58:52.309705 32179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.306145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:38289 (local address 127.29.112.193:37663)
0812 18:58:52.306434 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:52.306445 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:52.306548 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:52.306766 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:52.306769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:52.306776 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:58:52.306877 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.306888 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.307679 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.307682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.308428 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.308436 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.308832 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.308835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.308922 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.308960 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:58:52.309220 (+ 260us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:58:52.309227 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 18:58:52.309279 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 18:58:52.309347 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":84,"threads_started":1}
I20260812 18:58:52.315920 31867 raft_consensus.cc:1261] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5 [term 1 FOLLOWER]: Refusing update from remote peer d1fd088b71fa4c3493b47934a8547148: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:52.315925 32039 raft_consensus.cc:1261] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4 [term 1 FOLLOWER]: Refusing update from remote peer d1fd088b71fa4c3493b47934a8547148: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:52.316376 32157 consensus_queue.cc:1035] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [LEADER]: Connected to new peer: Peer: permanent_uuid: "154072e298524a10ad785ef932eb9de4" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:52.316543 32126 consensus_queue.cc:1035] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2fdfed0a948490089af8da4e7c352b5" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 38289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:52.320292 32137 mvcc.cc:205] Tried to move back new op lower bound from 7317754397961547776 to 7317754397312045056. Current Snapshot: MvccSnapshot[applied={T|T < 7317754397961547776}]
I20260812 18:58:52.344250 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.334789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:60559 (local address 127.29.112.193:43373)
0812 18:58:52.334896 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:52.334900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:52.334912 (+ 12us) server_negotiation.cc:407] Connection header received
0812 18:58:52.334984 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:52.334987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:52.335028 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:52.335094 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:52.335332 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.336086 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.342937 (+ 6851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.343286 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.343380 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.343785 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:52.343864 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:52.343907 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:52.344077 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 18:58:52.344133 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 18:58:52.344734 32184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.334468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:43373 (local address 127.29.112.194:60559)
0812 18:58:52.334782 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:52.334798 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:52.334896 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:52.335112 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:52.335114 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:52.335121 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:58:52.335209 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.335214 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.341874 (+ 6660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.341877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.342826 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:52.342834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.343457 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.343460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:52.343582 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.343635 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:58:52.343978 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:58:52.343983 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 18:58:52.344501 (+ 518us) client_negotiation.cc:240] Negotiation successful
0812 18:58:52.344584 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":103,"threads_started":1}
I20260812 18:58:52.365311 31514 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:52.365458 31514 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:52.367683 31514 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 487869a8c1d74f89ad5ab6e70545a589: Sending DeleteTablet for 3 replicas of tablet 736115cf295d4b8993f447ac41a59989
I20260812 18:58:52.368253 31677 tablet_service.cc:1514] Processing DeleteTablet for tablet 736115cf295d4b8993f447ac41a59989 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:52 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:45676
I20260812 18:58:52.368307 32014 tablet_service.cc:1514] Processing DeleteTablet for tablet 736115cf295d4b8993f447ac41a59989 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:52 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:55114
I20260812 18:58:52.368397 32150 tablet_replica.cc:330] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148: stopping tablet replica
I20260812 18:58:52.368487 32150 raft_consensus.cc:2229] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:58:52.368491 32146 tablet_replica.cc:330] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4: stopping tablet replica
I20260812 18:58:52.368618 32146 raft_consensus.cc:2229] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:52.368705 32150 raft_consensus.cc:2258] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:52.368705 32146 raft_consensus.cc:2258] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:52.369377 32146 ts_tablet_manager.cc:1905] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:52.369642 32150 ts_tablet_manager.cc:1905] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:52.370042 31846 tablet_service.cc:1514] Processing DeleteTablet for tablet 736115cf295d4b8993f447ac41a59989 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:52 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:33272
I20260812 18:58:52.370170 32148 tablet_replica.cc:330] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5: stopping tablet replica
I20260812 18:58:52.370266 32148 raft_consensus.cc:2229] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:52.370343 32148 raft_consensus.cc:2258] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:52.372143 32146 ts_tablet_manager.cc:1918] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:58:52.372216 32146 log.cc:1196] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/736115cf295d4b8993f447ac41a59989
I20260812 18:58:52.372483 32146 ts_tablet_manager.cc:1939] T 736115cf295d4b8993f447ac41a59989 P 154072e298524a10ad785ef932eb9de4: Deleting consensus metadata
I20260812 18:58:52.372970 32150 ts_tablet_manager.cc:1918] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:58:52.373035 32150 log.cc:1196] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-0/wal/wals/736115cf295d4b8993f447ac41a59989
I20260812 18:58:52.373104 31499 catalog_manager.cc:4841] TS 154072e298524a10ad785ef932eb9de4 (127.29.112.195:42851): tablet 736115cf295d4b8993f447ac41a59989 (table test-table [id=5e99bb6555fc4d06bbe1f89e85278d2a]) successfully deleted
I20260812 18:58:52.373252 32150 ts_tablet_manager.cc:1939] T 736115cf295d4b8993f447ac41a59989 P d1fd088b71fa4c3493b47934a8547148: Deleting consensus metadata
I20260812 18:58:52.373919 31496 catalog_manager.cc:4841] TS d1fd088b71fa4c3493b47934a8547148 (127.29.112.193:43373): tablet 736115cf295d4b8993f447ac41a59989 (table test-table [id=5e99bb6555fc4d06bbe1f89e85278d2a]) successfully deleted
I20260812 18:58:52.374486 32148 ts_tablet_manager.cc:1905] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:52.376915 32148 ts_tablet_manager.cc:1918] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:58:52.376989 32148 log.cc:1196] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/736115cf295d4b8993f447ac41a59989
I20260812 18:58:52.377373 32148 ts_tablet_manager.cc:1939] T 736115cf295d4b8993f447ac41a59989 P e2fdfed0a948490089af8da4e7c352b5: Deleting consensus metadata
I20260812 18:58:52.378077 31498 catalog_manager.cc:4841] TS e2fdfed0a948490089af8da4e7c352b5 (127.29.112.194:38289): tablet 736115cf295d4b8993f447ac41a59989 (table test-table [id=5e99bb6555fc4d06bbe1f89e85278d2a]) successfully deleted
Aug 12 18:58:52 dist-test-slave-6nmv krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.193@KRBTEST.COM
I20260812 18:58:52.387545 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.375383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.29.112.193:43373)
0812 18:58:52.375478 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:52.375481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:52.376006 (+ 525us) server_negotiation.cc:407] Connection header received
0812 18:58:52.376216 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:52.376219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:52.376258 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:52.376301 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:52.377376 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.378103 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.379202 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.379525 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.379587 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.383390 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:52.383399 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:52.383412 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:52.383436 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:52.385550 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:52.386055 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:52.386061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:52.386063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:52.386159 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:52.386476 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:52.386479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:52.386481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:52.386770 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:52.386964 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:52.387255 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 18:58:52.387373 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260812 18:58:52.387933 31752 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45682
I20260812 18:58:52.398154 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:52.389511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.29.112.254:43231)
0812 18:58:52.389620 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:52.389624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:52.390111 (+ 487us) server_negotiation.cc:407] Connection header received
0812 18:58:52.390188 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:52.390191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:52.390250 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:52.390327 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:52.391025 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.391759 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.392850 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:52.393172 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:52.393282 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:52.394770 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:52.394773 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:52.394776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:52.394814 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:52.396470 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:52.396908 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:52.396911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:52.396913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:52.397005 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:52.397284 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:52.397287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:52.397289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:52.397494 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:52.397621 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:52.397896 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 18:58:52.398018 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 18:58:52.399003 31514 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40316
I20260812 18:58:52.400236 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 31602
I20260812 18:58:52.411923 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 31769
I20260812 18:58:52.422950 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 31933
I20260812 18:58:52.431197 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 31476
2026-08-12T18:58:52Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2795 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:58:52 dist-test-slave-6nmv krb5kdc[32195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:58:52 dist-test-slave-6nmv krb5kdc[32195](info): set up 2 sockets
Aug 12 18:58:52 dist-test-slave-6nmv krb5kdc[32195](info): commencing operation
krb5kdc: starting...
W20260812 18:58:53.513446 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:58:53 dist-test-slave-6nmv krb5kdc[32195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:58:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:58:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:58:53.630424 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:39707
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:39707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:53.732659 32214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:53.732913 32214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:53.732965 32214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:53.736841 32214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:58:53.736889 32214 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 18:58:53.736905 32214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:53.736922 32214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:53.736937 32214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:58:53.736953 32214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:58:53.741966 32214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:39707
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:39707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:53.742872 32214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:53.743810 32214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:53.751999 32222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:53.751999 32220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:53.752079 32224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:53.752302 32214 server_base.cc:1034] running on GCE node
I20260812 18:58:53.752983 32214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:53.754144 32214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:53.755416 32214 hybrid_clock.cc:648] HybridClock initialized: now 1786561133755390 us; error 45 us; skew 500 ppm
Aug 12 18:58:53 dist-test-slave-6nmv krb5kdc[32195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561133, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:53.761868 32214 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:58:53.762814 32214 webserver.cc:460] Webserver started at http://127.29.112.254:41551/ using document root <none> and password file <none>
I20260812 18:58:53.763406 32214 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:53.763520 32214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:53.763741 32214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:53.765758 32214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "ad288562d7274a28a73731e961d362ff"
format_stamp: "Formatted at 2026-08-12 18:58:53 on dist-test-slave-6nmv"
server_key: "9ae53b374f5c3fce40337bb5d521f8f5"
server_key_iv: "262458accf0766eb753835ec06b8d408"
server_key_version: "encryptionkey@0"
I20260812 18:58:53.766426 32214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "ad288562d7274a28a73731e961d362ff"
format_stamp: "Formatted at 2026-08-12 18:58:53 on dist-test-slave-6nmv"
server_key: "9ae53b374f5c3fce40337bb5d521f8f5"
server_key_iv: "262458accf0766eb753835ec06b8d408"
server_key_version: "encryptionkey@0"
I20260812 18:58:53.769984 32214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 18:58:53.772234 32232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:53.773412 32214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:58:53.773545 32214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "ad288562d7274a28a73731e961d362ff"
format_stamp: "Formatted at 2026-08-12 18:58:53 on dist-test-slave-6nmv"
server_key: "9ae53b374f5c3fce40337bb5d521f8f5"
server_key_iv: "262458accf0766eb753835ec06b8d408"
server_key_version: "encryptionkey@0"
I20260812 18:58:53.773649 32214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:53.800137 32214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:53.802876 32214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:53.803103 32214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:53.811340 32214 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:39707
I20260812 18:58:53.811352 32307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:39707 every 8 connection(s)
I20260812 18:58:53.812275 32214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:58:53.814985 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:53.816036 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 32214
I20260812 18:58:53.816139 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:58:53.816411 30147 external_mini_cluster.cc:1428] Setting key b0cf111d657615e46a19519fff0bd2df
I20260812 18:58:53.821089 32308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff: Bootstrap starting.
I20260812 18:58:53.823539 32308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:53.824450 32308 log.cc:824] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff: Log is configured to *not* fsync() on all Append() calls
Aug 12 18:58:53 dist-test-slave-6nmv krb5kdc[32195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:53.826587 32308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff: No bootstrap required, opened a new log
I20260812 18:58:53.830474 32308 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad288562d7274a28a73731e961d362ff" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39707 } }
I20260812 18:58:53.830713 32308 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:53.830761 32308 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad288562d7274a28a73731e961d362ff, State: Initialized, Role: FOLLOWER
I20260812 18:58:53.831094 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:53.817790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59078 (local address 127.29.112.254:39707)
0812 18:58:53.818209 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:53.818224 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:53.818274 (+ 50us) server_negotiation.cc:407] Connection header received
0812 18:58:53.818870 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:53.818887 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:53.819169 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:53.819425 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:53.820299 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:53.821312 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:53.822393 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:53.822787 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:53.822891 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:53.825644 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:53.825649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:53.825663 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:53.825689 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:53.828362 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:53.828744 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:53.828750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:53.828755 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:53.828879 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:53.829120 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:53.829123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:53.829124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:53.829579 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:53.829782 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:53.830011 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 18:58:53.830268 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":139,"threads_started":1}
I20260812 18:58:53.831156 32308 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad288562d7274a28a73731e961d362ff" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39707 } }
I20260812 18:58:53.831285 32308 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:58:53.831346 32308 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:58:53.831426 32308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:53.832427 32308 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad288562d7274a28a73731e961d362ff" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39707 } }
I20260812 18:58:53.832751 32308 leader_election.cc:302] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad288562d7274a28a73731e961d362ff; no voters:
I20260812 18:58:53.833076 32308 leader_election.cc:288] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:58:53.833232 32317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:53.833472 32317 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [term 1 LEADER]: Becoming Leader. State: Replica: ad288562d7274a28a73731e961d362ff, State: Running, Role: LEADER
I20260812 18:58:53.833799 32317 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad288562d7274a28a73731e961d362ff" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39707 } }
I20260812 18:58:53.834285 32308 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:58:53.835611 32318 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad288562d7274a28a73731e961d362ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad288562d7274a28a73731e961d362ff" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39707 } } }
I20260812 18:58:53.835625 32319 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad288562d7274a28a73731e961d362ff. Latest consensus state: current_term: 1 leader_uuid: "ad288562d7274a28a73731e961d362ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad288562d7274a28a73731e961d362ff" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39707 } } }
I20260812 18:58:53.835726 32318 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [sys.catalog]: This master's current role is: LEADER
I20260812 18:58:53.835731 32319 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff [sys.catalog]: This master's current role is: LEADER
I20260812 18:58:53.836055 32329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:58:53.838497 32329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:58:53.843114 32329 catalog_manager.cc:1347] Generated new cluster ID: 880cd7ff245f458d8420b65f57cde392
I20260812 18:58:53.843168 32329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:58:53.849192 32329 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:58:53.850198 32329 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:58:53.862664 32329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff: Generated new TSK 0
I20260812 18:58:53.863263 32329 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 18:58:53.896836 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:39707
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:54.003057 32344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:54.003326 32344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:54.003369 32344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:54.007278 32344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:54.007328 32344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:54.007364 32344 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 18:58:54.007377 32344 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 18:58:54.007406 32344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:58:54.012364 32344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.254:39707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:54.013274 32344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:54.014256 32344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:54.023046 32353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:54.023164 32344 server_base.cc:1034] running on GCE node
W20260812 18:58:54.023110 32354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:54.023119 32358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:54.023674 32344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:54.024335 32344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:54.025519 32344 hybrid_clock.cc:648] HybridClock initialized: now 1786561134025512 us; error 30 us; skew 500 ppm
Aug 12 18:58:54 dist-test-slave-6nmv krb5kdc[32195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:54.028400 32344 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 18:58:54.029152 32344 webserver.cc:460] Webserver started at http://127.29.112.193:39891/ using document root <none> and password file <none>
I20260812 18:58:54.029687 32344 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:54.029795 32344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:54.030081 32344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:54.032049 32344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "47444a585872484187527e66e2a51d5b"
format_stamp: "Formatted at 2026-08-12 18:58:54 on dist-test-slave-6nmv"
server_key: "4ac4c01c81758655c4f75068f5e052fd"
server_key_iv: "a1e941993c5ebe3338877d3a4a48a544"
server_key_version: "encryptionkey@0"
I20260812 18:58:54.032593 32344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "47444a585872484187527e66e2a51d5b"
format_stamp: "Formatted at 2026-08-12 18:58:54 on dist-test-slave-6nmv"
server_key: "4ac4c01c81758655c4f75068f5e052fd"
server_key_iv: "a1e941993c5ebe3338877d3a4a48a544"
server_key_version: "encryptionkey@0"
I20260812 18:58:54.036299 32344 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260812 18:58:54.038625 32366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:54.039713 32344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 18:58:54.039829 32344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "47444a585872484187527e66e2a51d5b"
format_stamp: "Formatted at 2026-08-12 18:58:54 on dist-test-slave-6nmv"
server_key: "4ac4c01c81758655c4f75068f5e052fd"
server_key_iv: "a1e941993c5ebe3338877d3a4a48a544"
server_key_version: "encryptionkey@0"
I20260812 18:58:54.039945 32344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:54.053799 32344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:54.056511 32344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:54.056720 32344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:54.058115 32344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:54.058162 32344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:54.058220 32344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:54.058254 32344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 18:58:54 dist-test-slave-6nmv krb5kdc[32195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:54.069332 32344 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:39051
I20260812 18:58:54.069546 32506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:39051 every 8 connection(s)
I20260812 18:58:54.070422 32344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:58:54.072474 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 32344
I20260812 18:58:54.072590 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:58:54.072882 30147 external_mini_cluster.cc:1428] Setting key 60eeea36ab5fac7feedd7a42dfca78d7
I20260812 18:58:54.073127 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.058902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:36701 (local address 127.29.112.254:39707)
0812 18:58:54.059076 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.059083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.059826 (+ 743us) server_negotiation.cc:407] Connection header received
0812 18:58:54.060970 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.060977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.061057 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.061160 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:54.062514 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.063255 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.064484 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.064738 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.064855 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.068766 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:54.068770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:54.068772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:54.068800 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:54.070995 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.071502 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.071507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.071509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.071618 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.071916 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.071919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.071920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.072097 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:54.072176 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.072790 (+ 614us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.072953 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 18:58:54.074095 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.059216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:39707 (local address 127.29.112.193:36701)
0812 18:58:54.059669 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.059714 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.060749 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.061341 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.061350 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.061866 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:54.062370 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.062384 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.063386 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.063389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.064354 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.064364 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.064909 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.064914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.065146 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.065755 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:54.065783 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:54.068599 (+ 2816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:54.071093 (+ 2494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.071098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.071110 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.071395 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.071694 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.071696 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.071698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.071835 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.072225 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:54.072230 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:54.072464 (+ 234us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.072728 (+ 264us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.073074 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":142,"threads_started":1}
I20260812 18:58:54.075356 32507 heartbeater.cc:347] Connected to a master server at 127.29.112.254:39707
I20260812 18:58:54.075510 32507 heartbeater.cc:464] Registering TS with master...
I20260812 18:58:54.075984 32507 heartbeater.cc:511] Master 127.29.112.254:39707 requested a full tablet report, sending...
I20260812 18:58:54.077677 32252 ts_manager.cc:194] Registered new tserver with Master: 47444a585872484187527e66e2a51d5b (127.29.112.193:39051)
I20260812 18:58:54.079073 32252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:36701
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
I20260812 18:58:54.088488 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.077958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:49879 (local address 127.29.112.254:39707)
0812 18:58:54.079655 (+ 1697us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.079658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.079695 (+ 37us) server_negotiation.cc:407] Connection header received
0812 18:58:54.079729 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.079732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.079771 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.079858 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:54.082164 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.082650 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.083893 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.084108 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.084190 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.085326 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:54.085329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:54.085331 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:54.085350 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:54.086955 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.087316 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.087322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.087326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.087420 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.087655 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.087660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.087665 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.087891 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:54.087995 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.088272 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.088359 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1640}
I20260812 18:58:54.088615 32516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.078024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:39707 (local address 127.29.112.193:49879)
0812 18:58:54.078312 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.078327 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.078428 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.079902 (+ 1474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.079908 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.081887 (+ 1979us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:54.082036 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.082044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.082789 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.082795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.083754 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.083776 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.084221 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.084224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.084290 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.084876 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:54.084886 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:54.085207 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:54.087060 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.087062 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.087064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.087230 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.087503 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.087505 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.087506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.087575 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.087993 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:54.087998 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:54.088112 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.088220 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.088366 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":99,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 18:58:54.101289 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:39707
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:54.207017 32518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:54.207237 32518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:54.207276 32518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:54.211031 32518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:54.211078 32518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:54.211114 32518 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 18:58:54.211171 32518 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 18:58:54.211237 32518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:58:54.216208 32518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.254:39707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:54.217162 32518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:54.218135 32518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 18:58:54.227659 32518 server_base.cc:1034] running on GCE node
W20260812 18:58:54.227644 32526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:54.227644 32523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:54.227818 32524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:54.228216 32518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:54.228854 32518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:54.230006 32518 hybrid_clock.cc:648] HybridClock initialized: now 1786561134229977 us; error 43 us; skew 500 ppm
Aug 12 18:58:54 dist-test-slave-6nmv krb5kdc[32195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:54.232681 32518 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 18:58:54.233500 32518 webserver.cc:460] Webserver started at http://127.29.112.194:46873/ using document root <none> and password file <none>
I20260812 18:58:54.234149 32518 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:54.234242 32518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:54.234481 32518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:54.236689 32518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "0e4fec56450d4561b9d9c310f818f3cf"
format_stamp: "Formatted at 2026-08-12 18:58:54 on dist-test-slave-6nmv"
server_key: "c17371dfcd8272b0f131310a8e881073"
server_key_iv: "58c70dc3d51e7750943dd377828878ba"
server_key_version: "encryptionkey@0"
I20260812 18:58:54.237247 32518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "0e4fec56450d4561b9d9c310f818f3cf"
format_stamp: "Formatted at 2026-08-12 18:58:54 on dist-test-slave-6nmv"
server_key: "c17371dfcd8272b0f131310a8e881073"
server_key_iv: "58c70dc3d51e7750943dd377828878ba"
server_key_version: "encryptionkey@0"
I20260812 18:58:54.240898 32518 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 18:58:54.243460 32535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:54.244642 32518 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:58:54.244773 32518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "0e4fec56450d4561b9d9c310f818f3cf"
format_stamp: "Formatted at 2026-08-12 18:58:54 on dist-test-slave-6nmv"
server_key: "c17371dfcd8272b0f131310a8e881073"
server_key_iv: "58c70dc3d51e7750943dd377828878ba"
server_key_version: "encryptionkey@0"
I20260812 18:58:54.244882 32518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:54.277114 32518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:54.279999 32518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:54.280203 32518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:54.281612 32518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:54.281684 32518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:54.281754 32518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:54.281787 32518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 18:58:54 dist-test-slave-6nmv krb5kdc[32195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:54.292271 32518 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:42649
I20260812 18:58:54.292353 32671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:42649 every 8 connection(s)
I20260812 18:58:54.293147 32518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:58:54.295200 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.282846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:39707 (local address 127.29.112.194:46771)
0812 18:58:54.283277 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.283322 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.284251 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.284895 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.284907 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.285430 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:54.286158 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.286179 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.287044 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.287047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.287910 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.287918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.288435 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.288441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.288598 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.289158 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:54.289172 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:54.290706 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:54.292693 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.292699 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.292711 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.292994 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.293281 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.293285 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.293287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.293425 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.293898 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:54.293904 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:54.294162 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.294305 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.294533 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":120,"threads_started":1}
I20260812 18:58:54.295807 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.282462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:46771 (local address 127.29.112.254:39707)
0812 18:58:54.282606 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.282611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.283437 (+ 826us) server_negotiation.cc:407] Connection header received
0812 18:58:54.284515 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.284518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.284578 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.284703 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:54.286320 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.286921 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.288026 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.288264 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.288361 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.290860 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:54.290863 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:54.290865 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:54.290911 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:54.292583 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.293095 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.293098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.293100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.293179 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.293531 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.293534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.293535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.293745 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:54.295490 (+ 1745us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.295582 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.295687 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 18:58:54.296700 32672 heartbeater.cc:347] Connected to a master server at 127.29.112.254:39707
I20260812 18:58:54.296842 32672 heartbeater.cc:464] Registering TS with master...
I20260812 18:58:54.297288 32672 heartbeater.cc:511] Master 127.29.112.254:39707 requested a full tablet report, sending...
I20260812 18:58:54.297703 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 32518
I20260812 18:58:54.297860 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:58:54.298166 30147 external_mini_cluster.cc:1428] Setting key eb595bf5e7a8589adb1b1b20a4a23a59
I20260812 18:58:54.298352 32249 ts_manager.cc:194] Registered new tserver with Master: 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194:42649)
I20260812 18:58:54.299057 32249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:46771
I20260812 18:58:54.306227 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.298884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:48839 (local address 127.29.112.254:39707)
0812 18:58:54.299345 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.299355 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.299365 (+ 10us) server_negotiation.cc:407] Connection header received
0812 18:58:54.299388 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.299390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.299417 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.299501 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:54.299935 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.300400 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.301198 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.301456 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.301577 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.302905 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:54.302909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:54.302911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:54.302933 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:54.304439 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.304965 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.304981 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.304982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.305072 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.305433 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.305435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.305436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.305622 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:54.305698 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.306047 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.306132 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388}
I20260812 18:58:54.306450 32678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.298859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:39707 (local address 127.29.112.194:48839)
0812 18:58:54.299109 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.299125 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.299202 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.299517 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.299521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.299727 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:54.299827 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.299833 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.300525 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.300528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.301085 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.301093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.301706 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.301711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.301843 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.302318 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:54.302337 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:54.302774 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:54.304553 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.304559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.304563 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.304839 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.305208 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.305213 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.305217 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.305316 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.305720 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:54.305725 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:54.305883 (+ 158us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.306009 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.306169 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 18:58:54.324589 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:39707
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:58:54.430805 32681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:58:54.431030 32681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:58:54.431073 32681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:58:54.434942 32681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:58:54.434989 32681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:58:54.435027 32681 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 18:58:54.435041 32681 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 18:58:54.435071 32681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:58:54.440122 32681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.254:39707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:58:54.441082 32681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:58:54.442108 32681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:58:54.452965 32689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:54.452965 32687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:58:54.453013 32691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:58:54.453236 32681 server_base.cc:1034] running on GCE node
I20260812 18:58:54.453740 32681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:58:54.454394 32681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:58:54.455577 32681 hybrid_clock.cc:648] HybridClock initialized: now 1786561134455565 us; error 25 us; skew 500 ppm
Aug 12 18:58:54 dist-test-slave-6nmv krb5kdc[32195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:58:54.458256 32681 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 18:58:54.459098 32681 webserver.cc:460] Webserver started at http://127.29.112.195:40901/ using document root <none> and password file <none>
I20260812 18:58:54.459717 32681 fs_manager.cc:359] Metadata directory not provided
I20260812 18:58:54.459808 32681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:58:54.460072 32681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:58:54.462163 32681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "7616ad583b874b65a40dc2bb50caf4d0"
format_stamp: "Formatted at 2026-08-12 18:58:54 on dist-test-slave-6nmv"
server_key: "cb31dc639187273792bbad49e895226a"
server_key_iv: "99e12c4c817a9268b7d1df06f077f330"
server_key_version: "encryptionkey@0"
I20260812 18:58:54.462724 32681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "7616ad583b874b65a40dc2bb50caf4d0"
format_stamp: "Formatted at 2026-08-12 18:58:54 on dist-test-slave-6nmv"
server_key: "cb31dc639187273792bbad49e895226a"
server_key_iv: "99e12c4c817a9268b7d1df06f077f330"
server_key_version: "encryptionkey@0"
I20260812 18:58:54.466214 32681 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 18:58:54.468685 32698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:54.470077 32681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:58:54.470168 32681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "7616ad583b874b65a40dc2bb50caf4d0"
format_stamp: "Formatted at 2026-08-12 18:58:54 on dist-test-slave-6nmv"
server_key: "cb31dc639187273792bbad49e895226a"
server_key_iv: "99e12c4c817a9268b7d1df06f077f330"
server_key_version: "encryptionkey@0"
I20260812 18:58:54.470253 32681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:58:54.485203 32681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:58:54.487888 32681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:58:54.488039 32681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:58:54.489298 32681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:58:54.489370 32681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:58:54.489435 32681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:58:54.489497 32681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 18:58:54 dist-test-slave-6nmv krb5kdc[32195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:54.500814 32681 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:33543
I20260812 18:58:54.500902 365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:33543 every 8 connection(s)
I20260812 18:58:54.501748 32681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:58:54.503463 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.490264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:60423 (local address 127.29.112.254:39707)
0812 18:58:54.490404 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.490408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.491425 (+ 1017us) server_negotiation.cc:407] Connection header received
0812 18:58:54.492505 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.492509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.492574 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.492641 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:54.494404 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.495032 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.496150 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.496508 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.496624 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.499487 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:54.499491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:54.499493 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:54.499521 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:54.501137 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.501757 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.501762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.501764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.501919 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.502325 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.502329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.502331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.502597 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:54.502694 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.503163 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.503286 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 18:58:54.504186 32710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.490791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:39707 (local address 127.29.112.195:60423)
0812 18:58:54.491265 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.491303 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.492256 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.493131 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.493139 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.493546 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:54.494231 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.494249 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.495161 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.495165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.496018 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.496027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.496674 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.496679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.496900 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.497568 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:54.497596 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:54.499311 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:54.501304 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.501310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.501323 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.501646 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.502032 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.502035 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.502037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.502204 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.502688 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:54.502694 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:54.502932 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.503117 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.503419 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":133,"threads_started":1}
I20260812 18:58:54.505491 366 heartbeater.cc:347] Connected to a master server at 127.29.112.254:39707
I20260812 18:58:54.505646 366 heartbeater.cc:464] Registering TS with master...
I20260812 18:58:54.506212 366 heartbeater.cc:511] Master 127.29.112.254:39707 requested a full tablet report, sending...
I20260812 18:58:54.507261 32249 ts_manager.cc:194] Registered new tserver with Master: 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195:33543)
I20260812 18:58:54.507974 32249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:60423
I20260812 18:58:54.510068 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 32681
I20260812 18:58:54.510174 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:58:54.510462 30147 external_mini_cluster.cc:1428] Setting key e11bf649bbad0d1db8918763c2bf0840
I20260812 18:58:54.512641 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 18:58:54.514618 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.507575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:50815 (local address 127.29.112.254:39707)
0812 18:58:54.507689 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.507694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.507874 (+ 180us) server_negotiation.cc:407] Connection header received
0812 18:58:54.507988 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.507992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.508064 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.508147 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:54.508623 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.509134 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.510002 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.510298 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.510405 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.511541 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:54.511545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:54.511547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:54.511570 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:54.512997 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.513408 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.513411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.513413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.513493 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.513767 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.513771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.513773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.513999 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:54.514108 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.514391 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.514493 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 18:58:54.514628 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.507526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:39707 (local address 127.29.112.195:50815)
0812 18:58:54.507771 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.507788 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.507913 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.508175 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.508178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.508396 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:54.508521 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.508530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.509261 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.509264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.509877 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.509887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.510416 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.510420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.510516 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.510986 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:54.511000 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:54.511421 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:54.513102 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.513106 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.513108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.513301 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.513585 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.513589 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.513590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.513677 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.514109 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:54.514113 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:54.514207 (+ 94us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.514316 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.514452 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":89,"threads_started":1}
Aug 12 18:58:54 dist-test-slave-6nmv krb5kdc[32195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 18:58:54 dist-test-slave-6nmv krb5kdc[32195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:54.538918 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.530847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.29.112.254:39707)
0812 18:58:54.530966 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.530969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.530980 (+ 11us) server_negotiation.cc:407] Connection header received
0812 18:58:54.531019 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.531022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.531057 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.531147 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:54.531840 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.532346 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.533192 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.533400 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.533503 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.536066 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:54.536069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:54.536071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:54.536092 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:54.537302 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.537668 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.537671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.537672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.537773 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.538125 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.538127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.538128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.538289 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:54.538395 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.538595 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.538687 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 18:58:54.541730 32249 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:58:54.544926 32249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:58:54.559587 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.553430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42649 (local address 127.0.0.1:58842)
0812 18:58:54.553974 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.553989 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.554074 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.554962 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.554969 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.555018 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:54.555335 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.555342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.556732 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.556735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.558223 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.558234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.559193 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.559196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.559275 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.559304 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.559380 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.559463 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":296,"spinlock_wait_cycles":7040,"thread_start_us":100,"threads_started":1}
I20260812 18:58:54.559609 393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.554089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58842 (local address 127.29.112.194:42649)
0812 18:58:54.554432 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.554448 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.554468 (+ 20us) server_negotiation.cc:407] Connection header received
0812 18:58:54.554529 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.554533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.554671 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.554797 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:54.555490 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.556605 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.558336 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.559069 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.559220 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.559345 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.559403 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.559484 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":121,"threads_started":1}
I20260812 18:58:54.560305 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.553214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:39051 (local address 127.0.0.1:59458)
0812 18:58:54.553580 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.553629 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.553742 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.554962 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.554969 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.555017 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:54.555160 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.555167 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.556696 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.556699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.558178 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.558190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.559910 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.559913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.560004 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.560021 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.560106 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.560170 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":71,"spinlock_wait_cycles":5248,"thread_start_us":87,"threads_started":1}
I20260812 18:58:54.561079 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.554392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:33543 (local address 127.0.0.1:39056)
0812 18:58:54.554621 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.554638 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.554726 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.555639 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.555642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.555659 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:54.555785 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.555791 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.556962 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.556966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.558223 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.558234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.560648 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.560653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.560770 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.560791 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.560868 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.560949 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":66,"thread_start_us":115,"threads_started":1}
I20260812 18:58:54.561169 395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.554819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39056 (local address 127.29.112.195:33543)
0812 18:58:54.555153 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.555162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.555180 (+ 18us) server_negotiation.cc:407] Connection header received
0812 18:58:54.555268 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.555273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.555486 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.555925 (+ 439us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:54.556035 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.556850 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.559788 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.560518 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.560671 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.560795 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.560947 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.561040 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":130,"threads_started":1}
I20260812 18:58:54.562559 32589 tablet_service.cc:1467] Processing CreateTablet for tablet b49f6da789db4b69ba8d878b99a831c1 (DEFAULT_TABLE table=test-table [id=10cc387009c84d8bb427611d87fa8e2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:54.563437 32589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b49f6da789db4b69ba8d878b99a831c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:54.563184 32756 tablet_service.cc:1467] Processing CreateTablet for tablet b49f6da789db4b69ba8d878b99a831c1 (DEFAULT_TABLE table=test-table [id=10cc387009c84d8bb427611d87fa8e2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:54.563987 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b49f6da789db4b69ba8d878b99a831c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:54.564066 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.553393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59458 (local address 127.29.112.193:39051)
0812 18:58:54.554305 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.554321 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.554346 (+ 25us) server_negotiation.cc:407] Connection header received
0812 18:58:54.554427 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.554433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.554634 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.554792 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:54.555303 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.556555 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.558279 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.559081 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.559219 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.559345 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.560142 (+ 797us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.560232 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":800,"thread_start_us":86,"threads_started":1}
I20260812 18:58:54.566413 32426 tablet_service.cc:1467] Processing CreateTablet for tablet b49f6da789db4b69ba8d878b99a831c1 (DEFAULT_TABLE table=test-table [id=10cc387009c84d8bb427611d87fa8e2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:54.567473 32426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b49f6da789db4b69ba8d878b99a831c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:54.570500 396 tablet_bootstrap.cc:492] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: Bootstrap starting.
I20260812 18:58:54.571864 397 tablet_bootstrap.cc:492] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: Bootstrap starting.
I20260812 18:58:54.573025 396 tablet_bootstrap.cc:654] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:54.573920 397 tablet_bootstrap.cc:654] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:54.573976 396 log.cc:824] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:54.574777 397 log.cc:824] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:54.575088 399 tablet_bootstrap.cc:492] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: Bootstrap starting.
I20260812 18:58:54.576646 396 tablet_bootstrap.cc:492] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: No bootstrap required, opened a new log
I20260812 18:58:54.576848 396 ts_tablet_manager.cc:1397] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.002s
I20260812 18:58:54.577034 397 tablet_bootstrap.cc:492] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: No bootstrap required, opened a new log
I20260812 18:58:54.577209 397 ts_tablet_manager.cc:1397] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 18:58:54.579111 399 tablet_bootstrap.cc:654] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:54.579731 399 log.cc:824] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: Log is configured to *not* fsync() on all Append() calls
I20260812 18:58:54.580719 396 raft_consensus.cc:348] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.580801 397 raft_consensus.cc:348] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.580981 397 raft_consensus.cc:374] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:54.580981 396 raft_consensus.cc:374] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:54.581051 396 raft_consensus.cc:729] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ad583b874b65a40dc2bb50caf4d0, State: Initialized, Role: FOLLOWER
I20260812 18:58:54.581053 397 raft_consensus.cc:729] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4fec56450d4561b9d9c310f818f3cf, State: Initialized, Role: FOLLOWER
I20260812 18:58:54.581489 396 consensus_queue.cc:260] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.581489 397 consensus_queue.cc:260] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.582167 366 heartbeater.cc:503] Master 127.29.112.254:39707 was elected leader, sending a full tablet report...
I20260812 18:58:54.582468 396 ts_tablet_manager.cc:1428] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260812 18:58:54.582468 397 ts_tablet_manager.cc:1428] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260812 18:58:54.582787 32672 heartbeater.cc:503] Master 127.29.112.254:39707 was elected leader, sending a full tablet report...
I20260812 18:58:54.584263 399 tablet_bootstrap.cc:492] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: No bootstrap required, opened a new log
I20260812 18:58:54.584438 399 ts_tablet_manager.cc:1397] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260812 18:58:54.587947 399 raft_consensus.cc:348] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.588129 399 raft_consensus.cc:374] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:54.588200 399 raft_consensus.cc:729] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47444a585872484187527e66e2a51d5b, State: Initialized, Role: FOLLOWER
I20260812 18:58:54.588603 399 consensus_queue.cc:260] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.589229 32507 heartbeater.cc:503] Master 127.29.112.254:39707 was elected leader, sending a full tablet report...
I20260812 18:58:54.589413 399 ts_tablet_manager.cc:1428] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 18:58:54.740018 405 raft_consensus.cc:482] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:58:54.740237 405 raft_consensus.cc:504] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.741394 405 leader_election.cc:288] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47444a585872484187527e66e2a51d5b (127.29.112.193:39051), 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195:33543)
I20260812 18:58:54.745711 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.741669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:33543 (local address 127.29.112.194:34039)
0812 18:58:54.741785 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.741801 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.741939 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.742273 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.742276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.742301 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:54.742435 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.742445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.743462 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.743464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.744691 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.744702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.745350 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.745353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.745459 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.745492 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.745559 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.745615 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 18:58:54.745805 395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.741842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:34039 (local address 127.29.112.195:33543)
0812 18:58:54.742009 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.742016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.742031 (+ 15us) server_negotiation.cc:407] Connection header received
0812 18:58:54.742087 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.742092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.742145 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.742229 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:54.742595 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.743343 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.744835 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.745244 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.745335 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.745374 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.745610 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.745687 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 18:58:54.746412 310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b49f6da789db4b69ba8d878b99a831c1" candidate_uuid: "0e4fec56450d4561b9d9c310f818f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ad583b874b65a40dc2bb50caf4d0" is_pre_election: true
I20260812 18:58:54.746629 407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.741678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:39051 (local address 127.29.112.194:60669)
0812 18:58:54.742394 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.742409 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.742509 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.742828 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.742831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.742851 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:54.742993 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.742999 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.743908 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.743912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.745402 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.745410 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.746299 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.746302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.746368 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.746385 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.746441 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.746485 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":338,"thread_start_us":144,"threads_started":1}
I20260812 18:58:54.746656 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.741752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:60669 (local address 127.29.112.193:39051)
0812 18:58:54.741926 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.741936 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.742511 (+ 575us) server_negotiation.cc:407] Connection header received
0812 18:58:54.742633 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.742636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.742702 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.742778 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:54.743111 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.743765 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.745536 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.746183 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.746297 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.746339 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.746478 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.746546 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 18:58:54.746742 310 raft_consensus.cc:2454] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4fec56450d4561b9d9c310f818f3cf in term 0.
I20260812 18:58:54.747256 32536 leader_election.cc:302] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4fec56450d4561b9d9c310f818f3cf, 7616ad583b874b65a40dc2bb50caf4d0; no voters:
I20260812 18:58:54.747350 32452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b49f6da789db4b69ba8d878b99a831c1" candidate_uuid: "0e4fec56450d4561b9d9c310f818f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47444a585872484187527e66e2a51d5b" is_pre_election: true
I20260812 18:58:54.747562 405 raft_consensus.cc:2781] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:58:54.747602 32452 raft_consensus.cc:2454] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4fec56450d4561b9d9c310f818f3cf in term 0.
I20260812 18:58:54.747660 405 raft_consensus.cc:482] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:58:54.747718 405 raft_consensus.cc:3037] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:54.748834 405 raft_consensus.cc:504] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.749193 405 leader_election.cc:288] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [CANDIDATE]: Term 1 election: Requested vote from peers 47444a585872484187527e66e2a51d5b (127.29.112.193:39051), 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195:33543)
I20260812 18:58:54.749509 310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b49f6da789db4b69ba8d878b99a831c1" candidate_uuid: "0e4fec56450d4561b9d9c310f818f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ad583b874b65a40dc2bb50caf4d0"
I20260812 18:58:54.749523 32452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b49f6da789db4b69ba8d878b99a831c1" candidate_uuid: "0e4fec56450d4561b9d9c310f818f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47444a585872484187527e66e2a51d5b"
I20260812 18:58:54.749619 310 raft_consensus.cc:3037] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:54.749641 32452 raft_consensus.cc:3037] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:54.750612 310 raft_consensus.cc:2454] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4fec56450d4561b9d9c310f818f3cf in term 1.
I20260812 18:58:54.750628 32452 raft_consensus.cc:2454] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4fec56450d4561b9d9c310f818f3cf in term 1.
I20260812 18:58:54.750895 32536 leader_election.cc:302] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4fec56450d4561b9d9c310f818f3cf, 7616ad583b874b65a40dc2bb50caf4d0; no voters:
I20260812 18:58:54.751036 405 raft_consensus.cc:2781] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:54.751178 405 raft_consensus.cc:686] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 LEADER]: Becoming Leader. State: Replica: 0e4fec56450d4561b9d9c310f818f3cf, State: Running, Role: LEADER
I20260812 18:58:54.751479 405 consensus_queue.cc:237] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
W20260812 18:58:54.753026 367 tablet.cc:2369] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:58:54.754777 32249 catalog_manager.cc:5495] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194). New cstate: current_term: 1 leader_uuid: "0e4fec56450d4561b9d9c310f818f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } health_report { overall_health: UNKNOWN } } }
W20260812 18:58:54.794389 32673 tablet.cc:2369] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 18:58:54.821799 32508 tablet.cc:2369] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 18:58:54 dist-test-slave-6nmv krb5kdc[32195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561133, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:54.827519 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.819419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:39707 (local address 127.0.0.1:53640)
0812 18:58:54.819679 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.819700 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.819786 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.820007 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.820009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.820357 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:58:54.820453 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.820458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.821395 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.821397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.822105 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.822116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.822635 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.822637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.822695 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.823228 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:58:54.823239 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:58:54.824633 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:58:54.826286 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.826293 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.826297 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.826495 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.826714 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:58:54.826715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:58:54.826717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:58:54.826786 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:58:54.827087 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:58:54.827091 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:58:54.827176 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.827255 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.827355 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
I20260812 18:58:54.827520 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.819506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.29.112.254:39707)
0812 18:58:54.819631 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.819636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.819767 (+ 131us) server_negotiation.cc:407] Connection header received
0812 18:58:54.819850 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.819853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.819895 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.819979 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:54.820559 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.821296 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.822241 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.822538 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.822621 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.824753 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:54.824756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:54.824758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:54.824778 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:54.826201 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.826572 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.826575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.826576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.826642 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:54.826854 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:54.826856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:54.826857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:54.826998 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:54.827074 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.827291 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.827406 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":50}
I20260812 18:58:54.831036 32249 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:53640:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 18:58:54.831734 32249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:58:54.836386 32589 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0ac36128ec4e0ea0839ddc4f1918b8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e912d9f2a24422a95253c7b1f526bf3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 18:58:54.836422 32426 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0ac36128ec4e0ea0839ddc4f1918b8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e912d9f2a24422a95253c7b1f526bf3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 18:58:54.836515 32756 tablet_service.cc:1467] Processing CreateTablet for tablet 2a0ac36128ec4e0ea0839ddc4f1918b8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e912d9f2a24422a95253c7b1f526bf3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 18:58:54.837041 32589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0ac36128ec4e0ea0839ddc4f1918b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:54.837071 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0ac36128ec4e0ea0839ddc4f1918b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:54.837071 32426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a0ac36128ec4e0ea0839ddc4f1918b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:54.839365 397 tablet_bootstrap.cc:492] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: Bootstrap starting.
I20260812 18:58:54.839787 396 tablet_bootstrap.cc:492] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0: Bootstrap starting.
I20260812 18:58:54.840277 397 tablet_bootstrap.cc:654] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:54.840489 396 tablet_bootstrap.cc:654] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:54.841055 399 tablet_bootstrap.cc:492] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b: Bootstrap starting.
I20260812 18:58:54.841333 397 tablet_bootstrap.cc:492] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: No bootstrap required, opened a new log
I20260812 18:58:54.841398 397 ts_tablet_manager.cc:1397] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 18:58:54.841804 397 raft_consensus.cc:348] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.841946 397 raft_consensus.cc:374] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:54.841954 399 tablet_bootstrap.cc:654] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:54.841969 397 raft_consensus.cc:729] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4fec56450d4561b9d9c310f818f3cf, State: Initialized, Role: FOLLOWER
I20260812 18:58:54.842103 397 consensus_queue.cc:260] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.842362 397 ts_tablet_manager.cc:1428] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 18:58:54.842325 405 tablet_replica.cc:439] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } } }
I20260812 18:58:54.842434 405 tablet_replica.cc:442] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:54.843377 399 tablet_bootstrap.cc:492] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b: No bootstrap required, opened a new log
I20260812 18:58:54.843468 399 ts_tablet_manager.cc:1397] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 18:58:54.843885 399 raft_consensus.cc:348] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.844008 399 raft_consensus.cc:374] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:54.844065 399 raft_consensus.cc:729] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47444a585872484187527e66e2a51d5b, State: Initialized, Role: FOLLOWER
I20260812 18:58:54.844183 399 consensus_queue.cc:260] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.844457 399 ts_tablet_manager.cc:1428] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 18:58:54.844475 406 tablet_replica.cc:439] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } } }
I20260812 18:58:54.844653 406 tablet_replica.cc:442] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:54.845062 396 tablet_bootstrap.cc:492] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0: No bootstrap required, opened a new log
I20260812 18:58:54.845160 396 ts_tablet_manager.cc:1397] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 18:58:54.845551 396 raft_consensus.cc:348] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.845664 396 raft_consensus.cc:374] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:54.845693 396 raft_consensus.cc:729] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ad583b874b65a40dc2bb50caf4d0, State: Initialized, Role: FOLLOWER
I20260812 18:58:54.845824 396 consensus_queue.cc:260] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.846078 396 ts_tablet_manager.cc:1428] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:58:54.846071 404 tablet_replica.cc:439] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } } }
I20260812 18:58:54.846190 404 tablet_replica.cc:442] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:54.874581 405 raft_consensus.cc:482] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:58:54.874670 405 raft_consensus.cc:504] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.874985 405 leader_election.cc:288] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47444a585872484187527e66e2a51d5b (127.29.112.193:39051), 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195:33543)
I20260812 18:58:54.875388 310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0ac36128ec4e0ea0839ddc4f1918b8" candidate_uuid: "0e4fec56450d4561b9d9c310f818f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ad583b874b65a40dc2bb50caf4d0" is_pre_election: true
I20260812 18:58:54.875402 32452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0ac36128ec4e0ea0839ddc4f1918b8" candidate_uuid: "0e4fec56450d4561b9d9c310f818f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47444a585872484187527e66e2a51d5b" is_pre_election: true
I20260812 18:58:54.875531 310 raft_consensus.cc:2454] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4fec56450d4561b9d9c310f818f3cf in term 0.
I20260812 18:58:54.875542 32452 raft_consensus.cc:2454] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4fec56450d4561b9d9c310f818f3cf in term 0.
I20260812 18:58:54.875852 32536 leader_election.cc:302] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0e4fec56450d4561b9d9c310f818f3cf, 47444a585872484187527e66e2a51d5b, 7616ad583b874b65a40dc2bb50caf4d0; no voters:
I20260812 18:58:54.876003 405 raft_consensus.cc:2781] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:58:54.876060 405 raft_consensus.cc:482] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:58:54.876102 405 raft_consensus.cc:3037] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:54.876917 405 raft_consensus.cc:504] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.877197 405 leader_election.cc:288] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [CANDIDATE]: Term 1 election: Requested vote from peers 47444a585872484187527e66e2a51d5b (127.29.112.193:39051), 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195:33543)
I20260812 18:58:54.877497 32452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0ac36128ec4e0ea0839ddc4f1918b8" candidate_uuid: "0e4fec56450d4561b9d9c310f818f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47444a585872484187527e66e2a51d5b"
I20260812 18:58:54.877579 32452 raft_consensus.cc:3037] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:54.877755 310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a0ac36128ec4e0ea0839ddc4f1918b8" candidate_uuid: "0e4fec56450d4561b9d9c310f818f3cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ad583b874b65a40dc2bb50caf4d0"
I20260812 18:58:54.877897 310 raft_consensus.cc:3037] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:54.878705 32452 raft_consensus.cc:2454] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4fec56450d4561b9d9c310f818f3cf in term 1.
I20260812 18:58:54.878868 310 raft_consensus.cc:2454] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4fec56450d4561b9d9c310f818f3cf in term 1.
I20260812 18:58:54.879020 32539 leader_election.cc:302] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4fec56450d4561b9d9c310f818f3cf, 47444a585872484187527e66e2a51d5b; no voters:
I20260812 18:58:54.879226 405 raft_consensus.cc:2781] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:54.879323 405 raft_consensus.cc:686] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 LEADER]: Becoming Leader. State: Replica: 0e4fec56450d4561b9d9c310f818f3cf, State: Running, Role: LEADER
I20260812 18:58:54.879467 405 consensus_queue.cc:237] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.880192 405 tablet_replica.cc:439] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: TxnStatusTablet state changed. Reason: New leader 0e4fec56450d4561b9d9c310f818f3cf. Latest consensus state: current_term: 1 leader_uuid: "0e4fec56450d4561b9d9c310f818f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } } }
I20260812 18:58:54.880295 405 tablet_replica.cc:442] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: This TxnStatusTablet replica's current role is: LEADER
I20260812 18:58:54.880604 422 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 18:58:54.880839 422 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 18:58:54.881186 32252 catalog_manager.cc:5495] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194). New cstate: current_term: 1 leader_uuid: "0e4fec56450d4561b9d9c310f818f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } health_report { overall_health: UNKNOWN } } }
I20260812 18:58:54.899333 393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.895614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.29.112.194:42649)
0812 18:58:54.895737 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.895740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.895752 (+ 12us) server_negotiation.cc:407] Connection header received
0812 18:58:54.895913 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.895915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.895961 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.896032 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:54.896275 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.896706 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.897852 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.898170 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.898260 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.898563 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:54.898944 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:54.899040 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.899176 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.899229 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 18:58:54.899356 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.895512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42649 (local address 127.0.0.1:58848)
0812 18:58:54.895673 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.895688 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.895777 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.896074 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.896077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.896084 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:58:54.896179 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.896185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.896800 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.896803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.897685 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.897698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.898288 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.898292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.898378 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.898426 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:58:54.899046 (+ 620us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:58:54.899060 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.899145 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.899229 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 18:58:54.904562 310 raft_consensus.cc:1261] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Refusing update from remote peer 0e4fec56450d4561b9d9c310f818f3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:54.904605 32452 raft_consensus.cc:1261] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Refusing update from remote peer 0e4fec56450d4561b9d9c310f818f3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:54.905272 405 consensus_queue.cc:1035] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:54.905469 408 consensus_queue.cc:1035] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:54.909346 404 tablet_replica.cc:439] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0: TxnStatusTablet state changed. Reason: New leader 0e4fec56450d4561b9d9c310f818f3cf. Latest consensus state: current_term: 1 leader_uuid: "0e4fec56450d4561b9d9c310f818f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } } }
I20260812 18:58:54.909461 404 tablet_replica.cc:442] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:54.909839 406 tablet_replica.cc:439] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b: TxnStatusTablet state changed. Reason: New leader 0e4fec56450d4561b9d9c310f818f3cf. Latest consensus state: current_term: 1 leader_uuid: "0e4fec56450d4561b9d9c310f818f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } } }
I20260812 18:58:54.909965 406 tablet_replica.cc:442] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:54.911044 408 tablet_replica.cc:439] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0e4fec56450d4561b9d9c310f818f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } } }
I20260812 18:58:54.911137 408 tablet_replica.cc:442] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: This TxnStatusTablet replica's current role is: LEADER
I20260812 18:58:54.911281 404 tablet_replica.cc:439] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0e4fec56450d4561b9d9c310f818f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } } }
I20260812 18:58:54.911363 404 tablet_replica.cc:442] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 7616ad583b874b65a40dc2bb50caf4d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:54.913127 425 mvcc.cc:205] Tried to move back new op lower bound from 7317754408562028544 to 7317754408467894272. Current Snapshot: MvccSnapshot[applied={T|T < 7317754408562028544}]
I20260812 18:58:54.913363 426 mvcc.cc:205] Tried to move back new op lower bound from 7317754408562028544 to 7317754408467894272. Current Snapshot: MvccSnapshot[applied={T|T < 7317754408562028544}]
I20260812 18:58:54.913615 408 tablet_replica.cc:439] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0e4fec56450d4561b9d9c310f818f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } } }
I20260812 18:58:54.913703 408 tablet_replica.cc:442] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 0e4fec56450d4561b9d9c310f818f3cf: This TxnStatusTablet replica's current role is: LEADER
I20260812 18:58:54.914955 406 tablet_replica.cc:439] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0e4fec56450d4561b9d9c310f818f3cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } } }
I20260812 18:58:54.915062 406 tablet_replica.cc:442] T 2a0ac36128ec4e0ea0839ddc4f1918b8 P 47444a585872484187527e66e2a51d5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:58:54.916507 427 mvcc.cc:205] Tried to move back new op lower bound from 7317754408562028544 to 7317754408467894272. Current Snapshot: MvccSnapshot[applied={T|T < 7317754408562028544}]
I20260812 18:58:54.924338 393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.921001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.29.112.194:42649)
0812 18:58:54.921098 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.921118 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.921199 (+ 81us) server_negotiation.cc:407] Connection header received
0812 18:58:54.921309 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.921311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.921342 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.921434 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:54.921671 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.922187 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.923383 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.923581 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.923657 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.923946 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:54.924006 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:54.924058 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.924174 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.924211 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 18:58:54.934494 393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.931008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:38415 (local address 127.29.112.194:42649)
0812 18:58:54.931116 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:54.931120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:54.931278 (+ 158us) server_negotiation.cc:407] Connection header received
0812 18:58:54.931350 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:54.931352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:54.931400 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:54.931471 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:54.931693 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.932426 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.933230 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.933566 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.933664 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.934058 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:54.934133 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:54.934205 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:54.934290 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 18:58:54.934344 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":42}
I20260812 18:58:54.934499 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:54.930959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42649 (local address 127.29.112.194:38415)
0812 18:58:54.931194 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:54.931208 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:54.931292 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:54.931484 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:54.931486 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:54.931502 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:58:54.931578 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.931581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.932527 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.932529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.933127 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:54.933134 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:54.933673 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:54.933676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:54.933751 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:54.933857 (+ 106us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:58:54.934221 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:58:54.934227 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 18:58:54.934281 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 18:58:54.934339 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260812 18:58:54.940716 310 raft_consensus.cc:1261] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Refusing update from remote peer 0e4fec56450d4561b9d9c310f818f3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:54.940817 32452 raft_consensus.cc:1261] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Refusing update from remote peer 0e4fec56450d4561b9d9c310f818f3cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:54.941124 405 consensus_queue.cc:1035] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:54.941246 433 consensus_queue.cc:1035] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:54.982244 32252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:54.982453 32252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:54.984678 32252 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff: Sending DeleteTablet for 3 replicas of tablet b49f6da789db4b69ba8d878b99a831c1
I20260812 18:58:54.985337 32589 tablet_service.cc:1514] Processing DeleteTablet for tablet b49f6da789db4b69ba8d878b99a831c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:54 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:58842
I20260812 18:58:54.985337 32426 tablet_service.cc:1514] Processing DeleteTablet for tablet b49f6da789db4b69ba8d878b99a831c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:54 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:59458
I20260812 18:58:54.985504 32756 tablet_service.cc:1514] Processing DeleteTablet for tablet b49f6da789db4b69ba8d878b99a831c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:54 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:39056
I20260812 18:58:54.985800 447 tablet_replica.cc:330] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: stopping tablet replica
I20260812 18:58:54.985991 448 tablet_replica.cc:330] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: stopping tablet replica
I20260812 18:58:54.986207 447 raft_consensus.cc:2229] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:54.986268 444 tablet_replica.cc:330] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: stopping tablet replica
I20260812 18:58:54.986479 447 raft_consensus.cc:2258] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:54.986485 444 raft_consensus.cc:2229] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:58:54.986466 32252 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 18:58:54.986752 444 raft_consensus.cc:2258] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 18:58:54.986812 32252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:58:54.987875 448 raft_consensus.cc:2229] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:54.988066 448 raft_consensus.cc:2258] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:54.988099 444 ts_tablet_manager.cc:1905] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:54.989436 448 ts_tablet_manager.cc:1905] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:54.990923 447 ts_tablet_manager.cc:1905] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:54.991032 444 ts_tablet_manager.cc:1918] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:58:54.991123 444 log.cc:1196] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/b49f6da789db4b69ba8d878b99a831c1
I20260812 18:58:54.991473 444 ts_tablet_manager.cc:1939] T b49f6da789db4b69ba8d878b99a831c1 P 0e4fec56450d4561b9d9c310f818f3cf: Deleting consensus metadata
I20260812 18:58:54.991981 32756 tablet_service.cc:1467] Processing CreateTablet for tablet 9f65dfda6caf4374ad4bba4bcf8e34db (DEFAULT_TABLE table=test-table [id=fbfa2a5dd0e04054a07ac61c0efd9c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:54.992164 32426 tablet_service.cc:1467] Processing CreateTablet for tablet 9f65dfda6caf4374ad4bba4bcf8e34db (DEFAULT_TABLE table=test-table [id=fbfa2a5dd0e04054a07ac61c0efd9c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:54.992220 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f65dfda6caf4374ad4bba4bcf8e34db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:54.992246 32589 tablet_service.cc:1467] Processing CreateTablet for tablet 9f65dfda6caf4374ad4bba4bcf8e34db (DEFAULT_TABLE table=test-table [id=fbfa2a5dd0e04054a07ac61c0efd9c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:54.992391 32426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f65dfda6caf4374ad4bba4bcf8e34db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:54.992460 32589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f65dfda6caf4374ad4bba4bcf8e34db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:54.992537 32234 catalog_manager.cc:4841] TS 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194:42649): tablet b49f6da789db4b69ba8d878b99a831c1 (table test-table [id=10cc387009c84d8bb427611d87fa8e2b]) successfully deleted
I20260812 18:58:54.994426 396 tablet_bootstrap.cc:492] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0: Bootstrap starting.
I20260812 18:58:54.994479 399 tablet_bootstrap.cc:492] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b: Bootstrap starting.
I20260812 18:58:54.995424 396 tablet_bootstrap.cc:654] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:54.995434 399 tablet_bootstrap.cc:654] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:54.996011 397 tablet_bootstrap.cc:492] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf: Bootstrap starting.
I20260812 18:58:54.996436 447 ts_tablet_manager.cc:1918] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:58:54.996546 447 log.cc:1196] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/wal/wals/b49f6da789db4b69ba8d878b99a831c1
I20260812 18:58:54.996735 448 ts_tablet_manager.cc:1918] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:58:54.996847 448 log.cc:1196] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/b49f6da789db4b69ba8d878b99a831c1
I20260812 18:58:54.996948 397 tablet_bootstrap.cc:654] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:54.996984 447 ts_tablet_manager.cc:1939] T b49f6da789db4b69ba8d878b99a831c1 P 47444a585872484187527e66e2a51d5b: Deleting consensus metadata
I20260812 18:58:54.997169 448 ts_tablet_manager.cc:1939] T b49f6da789db4b69ba8d878b99a831c1 P 7616ad583b874b65a40dc2bb50caf4d0: Deleting consensus metadata
I20260812 18:58:54.998329 32233 catalog_manager.cc:4841] TS 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195:33543): tablet b49f6da789db4b69ba8d878b99a831c1 (table test-table [id=10cc387009c84d8bb427611d87fa8e2b]) successfully deleted
I20260812 18:58:54.998343 32236 catalog_manager.cc:4841] TS 47444a585872484187527e66e2a51d5b (127.29.112.193:39051): tablet b49f6da789db4b69ba8d878b99a831c1 (table test-table [id=10cc387009c84d8bb427611d87fa8e2b]) successfully deleted
I20260812 18:58:54.998795 397 tablet_bootstrap.cc:492] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf: No bootstrap required, opened a new log
I20260812 18:58:54.998895 397 ts_tablet_manager.cc:1397] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 18:58:54.999236 397 raft_consensus.cc:348] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.999367 397 raft_consensus.cc:374] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:54.999424 397 raft_consensus.cc:729] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4fec56450d4561b9d9c310f818f3cf, State: Initialized, Role: FOLLOWER
I20260812 18:58:54.999621 397 consensus_queue.cc:260] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:54.999882 397 ts_tablet_manager.cc:1428] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 18:58:55.001019 399 tablet_bootstrap.cc:492] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b: No bootstrap required, opened a new log
I20260812 18:58:55.001113 399 ts_tablet_manager.cc:1397] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260812 18:58:55.001439 399 raft_consensus.cc:348] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:55.001543 399 raft_consensus.cc:374] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:55.001580 399 raft_consensus.cc:729] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47444a585872484187527e66e2a51d5b, State: Initialized, Role: FOLLOWER
I20260812 18:58:55.001701 399 consensus_queue.cc:260] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:55.001751 396 tablet_bootstrap.cc:492] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0: No bootstrap required, opened a new log
I20260812 18:58:55.001971 399 ts_tablet_manager.cc:1428] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 18:58:55.002138 396 ts_tablet_manager.cc:1397] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260812 18:58:55.002538 396 raft_consensus.cc:348] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:55.002640 396 raft_consensus.cc:374] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:55.002677 396 raft_consensus.cc:729] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ad583b874b65a40dc2bb50caf4d0, State: Initialized, Role: FOLLOWER
I20260812 18:58:55.002823 396 consensus_queue.cc:260] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:55.003058 396 ts_tablet_manager.cc:1428] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:58:55.004842 404 raft_consensus.cc:482] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:58:55.004959 404 raft_consensus.cc:504] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:55.006083 404 leader_election.cc:288] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47444a585872484187527e66e2a51d5b (127.29.112.193:39051), 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194:42649)
I20260812 18:58:55.011094 393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:55.006544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:56945 (local address 127.29.112.194:42649)
0812 18:58:55.006734 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:55.006738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:55.006987 (+ 249us) server_negotiation.cc:407] Connection header received
0812 18:58:55.007189 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:55.007192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:55.007251 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:55.007327 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:55.007684 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.008381 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.010130 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.010493 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.010631 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:55.010690 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:55.010875 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 18:58:55.010952 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 18:58:55.011257 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:55.006594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:33453 (local address 127.29.112.193:39051)
0812 18:58:55.006713 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:55.006717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:55.006729 (+ 12us) server_negotiation.cc:407] Connection header received
0812 18:58:55.006781 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:55.006784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:55.006824 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:55.006899 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:55.007524 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.008330 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.010115 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.010677 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.010794 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:55.010839 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:55.011078 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 18:58:55.011158 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 18:58:55.011431 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:55.006312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42649 (local address 127.29.112.195:56945)
0812 18:58:55.006902 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:55.006917 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:55.007049 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:55.007436 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:55.007442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:55.007462 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:55.007570 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:55.007576 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.008524 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.008527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:55.009992 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:55.010002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.010624 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.010627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:55.010726 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:55.010758 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 18:58:55.011207 (+ 449us) client_negotiation.cc:240] Negotiation successful
0812 18:58:55.011288 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":104,"thread_start_us":75,"threads_started":1}
I20260812 18:58:55.011926 32616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f65dfda6caf4374ad4bba4bcf8e34db" candidate_uuid: "7616ad583b874b65a40dc2bb50caf4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4fec56450d4561b9d9c310f818f3cf" is_pre_election: true
I20260812 18:58:55.012133 32616 raft_consensus.cc:2454] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ad583b874b65a40dc2bb50caf4d0 in term 0.
I20260812 18:58:55.012334 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:55.006215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:39051 (local address 127.29.112.195:33453)
0812 18:58:55.006476 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:55.006491 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:55.006577 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:55.007035 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:55.007038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:55.007057 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:55.007396 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:55.007406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.008477 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.008483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:55.009992 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:55.010002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.010799 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.010805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:55.010942 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:55.010965 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 18:58:55.011848 (+ 883us) client_negotiation.cc:240] Negotiation successful
0812 18:58:55.011941 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":157,"thread_start_us":121,"threads_started":1}
I20260812 18:58:55.012571 32701 leader_election.cc:302] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4fec56450d4561b9d9c310f818f3cf, 7616ad583b874b65a40dc2bb50caf4d0; no voters:
I20260812 18:58:55.012722 32452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f65dfda6caf4374ad4bba4bcf8e34db" candidate_uuid: "7616ad583b874b65a40dc2bb50caf4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47444a585872484187527e66e2a51d5b" is_pre_election: true
I20260812 18:58:55.012830 404 raft_consensus.cc:2781] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:58:55.012861 32452 raft_consensus.cc:2454] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ad583b874b65a40dc2bb50caf4d0 in term 0.
I20260812 18:58:55.012895 404 raft_consensus.cc:482] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:58:55.012920 404 raft_consensus.cc:3037] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:55.013880 404 raft_consensus.cc:504] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:55.014243 404 leader_election.cc:288] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [CANDIDATE]: Term 1 election: Requested vote from peers 47444a585872484187527e66e2a51d5b (127.29.112.193:39051), 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194:42649)
I20260812 18:58:55.014546 32452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f65dfda6caf4374ad4bba4bcf8e34db" candidate_uuid: "7616ad583b874b65a40dc2bb50caf4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47444a585872484187527e66e2a51d5b"
I20260812 18:58:55.014568 32616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f65dfda6caf4374ad4bba4bcf8e34db" candidate_uuid: "7616ad583b874b65a40dc2bb50caf4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4fec56450d4561b9d9c310f818f3cf"
I20260812 18:58:55.014665 32452 raft_consensus.cc:3037] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:55.014683 32616 raft_consensus.cc:3037] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:55.015690 32616 raft_consensus.cc:2454] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ad583b874b65a40dc2bb50caf4d0 in term 1.
I20260812 18:58:55.015810 32452 raft_consensus.cc:2454] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ad583b874b65a40dc2bb50caf4d0 in term 1.
I20260812 18:58:55.016001 32701 leader_election.cc:302] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4fec56450d4561b9d9c310f818f3cf, 7616ad583b874b65a40dc2bb50caf4d0; no voters:
I20260812 18:58:55.016410 404 raft_consensus.cc:2781] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:55.016593 404 raft_consensus.cc:686] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 LEADER]: Becoming Leader. State: Replica: 7616ad583b874b65a40dc2bb50caf4d0, State: Running, Role: LEADER
I20260812 18:58:55.016819 404 consensus_queue.cc:237] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:55.019084 32249 catalog_manager.cc:5495] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195). New cstate: current_term: 1 leader_uuid: "7616ad583b874b65a40dc2bb50caf4d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } health_report { overall_health: HEALTHY } } }
I20260812 18:58:55.036264 395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:55.032582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39072 (local address 127.29.112.195:33543)
0812 18:58:55.032683 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:55.032689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:55.032723 (+ 34us) server_negotiation.cc:407] Connection header received
0812 18:58:55.032937 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:55.032940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:55.032994 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:55.033097 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:55.033323 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.033797 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.034833 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.035199 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.035288 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:55.035538 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:55.035920 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:55.036033 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:55.036103 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 18:58:55.036148 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 18:58:55.046103 459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:55.042559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42649 (local address 127.29.112.195:36409)
0812 18:58:55.042833 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:55.042846 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:55.042947 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:55.043169 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:55.043171 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:55.043176 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:58:55.043244 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:55.043247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.044207 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.044209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:55.044831 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:55.044840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.045430 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.045434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:55.045509 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:55.045544 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:58:55.045857 (+ 313us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:58:55.045863 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 18:58:55.045944 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 18:58:55.046005 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":97,"threads_started":1}
I20260812 18:58:55.046139 393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:55.042642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:36409 (local address 127.29.112.194:42649)
0812 18:58:55.042783 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:55.042788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:55.042914 (+ 126us) server_negotiation.cc:407] Connection header received
0812 18:58:55.043028 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:55.043031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:55.043073 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:55.043158 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:55.043338 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.044084 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.044946 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.045289 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.045402 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:55.045660 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:55.045739 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:55.045829 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:55.045972 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 18:58:55.046021 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 18:58:55.051574 32616 raft_consensus.cc:1261] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Refusing update from remote peer 7616ad583b874b65a40dc2bb50caf4d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:55.052039 32452 raft_consensus.cc:1261] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Refusing update from remote peer 7616ad583b874b65a40dc2bb50caf4d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:55.052143 455 consensus_queue.cc:1035] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:55.052378 404 consensus_queue.cc:1035] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:55.069883 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:55.065929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:33543 (local address 127.29.112.194:44499)
0812 18:58:55.066095 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:55.066110 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:55.066183 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:55.066627 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:55.066630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:55.066637 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:58:55.066720 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:55.066724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.067725 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.067727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:55.068276 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:55.068284 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.069025 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.069029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:55.069116 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:55.069175 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:58:55.069569 (+ 394us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:58:55.069577 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 18:58:55.069650 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 18:58:55.069732 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 18:58:55.070060 395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:55.066311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:44499 (local address 127.29.112.195:33543)
0812 18:58:55.066420 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:55.066424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:55.066436 (+ 12us) server_negotiation.cc:407] Connection header received
0812 18:58:55.066484 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:55.066487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:55.066526 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:55.066588 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:55.066811 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.067608 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.068502 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:55.068839 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:55.068944 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:55.069366 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:55.069467 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:55.069559 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:55.069887 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 18:58:55.069940 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 18:58:55.070804 426 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 18:58:56.637514 32248 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:56.637661 32248 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:56.640219 32248 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff: Sending DeleteTablet for 3 replicas of tablet 9f65dfda6caf4374ad4bba4bcf8e34db
I20260812 18:58:56.640906 32756 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f65dfda6caf4374ad4bba4bcf8e34db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:56 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:39056
I20260812 18:58:56.640913 32589 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f65dfda6caf4374ad4bba4bcf8e34db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:56 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:58842
I20260812 18:58:56.640933 32426 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f65dfda6caf4374ad4bba4bcf8e34db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:56 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:59458
I20260812 18:58:56.641283 493 tablet_replica.cc:330] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0: stopping tablet replica
I20260812 18:58:56.641283 495 tablet_replica.cc:330] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf: stopping tablet replica
I20260812 18:58:56.641413 493 raft_consensus.cc:2229] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:58:56.641422 495 raft_consensus.cc:2229] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:56.641547 495 raft_consensus.cc:2258] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:56.641710 493 raft_consensus.cc:2258] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:56.641973 494 tablet_replica.cc:330] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b: stopping tablet replica
I20260812 18:58:56.642102 494 raft_consensus.cc:2229] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:56.642198 494 raft_consensus.cc:2258] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:56.642472 493 ts_tablet_manager.cc:1905] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:56.643275 494 ts_tablet_manager.cc:1905] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:56.645892 493 ts_tablet_manager.cc:1918] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 18:58:56.645984 493 log.cc:1196] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/9f65dfda6caf4374ad4bba4bcf8e34db
I20260812 18:58:56.646119 494 ts_tablet_manager.cc:1918] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 18:58:56.646274 494 log.cc:1196] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/wal/wals/9f65dfda6caf4374ad4bba4bcf8e34db
I20260812 18:58:56.646278 495 ts_tablet_manager.cc:1905] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:56.646325 493 ts_tablet_manager.cc:1939] T 9f65dfda6caf4374ad4bba4bcf8e34db P 7616ad583b874b65a40dc2bb50caf4d0: Deleting consensus metadata
I20260812 18:58:56.646647 494 ts_tablet_manager.cc:1939] T 9f65dfda6caf4374ad4bba4bcf8e34db P 47444a585872484187527e66e2a51d5b: Deleting consensus metadata
I20260812 18:58:56.647049 32233 catalog_manager.cc:4841] TS 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195:33543): tablet 9f65dfda6caf4374ad4bba4bcf8e34db (table test-table [id=fbfa2a5dd0e04054a07ac61c0efd9c9f]) successfully deleted
I20260812 18:58:56.647375 32236 catalog_manager.cc:4841] TS 47444a585872484187527e66e2a51d5b (127.29.112.193:39051): tablet 9f65dfda6caf4374ad4bba4bcf8e34db (table test-table [id=fbfa2a5dd0e04054a07ac61c0efd9c9f]) successfully deleted
I20260812 18:58:56.648931 495 ts_tablet_manager.cc:1918] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 18:58:56.649021 495 log.cc:1196] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/9f65dfda6caf4374ad4bba4bcf8e34db
I20260812 18:58:56.649420 495 ts_tablet_manager.cc:1939] T 9f65dfda6caf4374ad4bba4bcf8e34db P 0e4fec56450d4561b9d9c310f818f3cf: Deleting consensus metadata
I20260812 18:58:56.650207 32234 catalog_manager.cc:4841] TS 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194:42649): tablet 9f65dfda6caf4374ad4bba4bcf8e34db (table test-table [id=fbfa2a5dd0e04054a07ac61c0efd9c9f]) successfully deleted
Aug 12 18:58:56 dist-test-slave-6nmv krb5kdc[32195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561136, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 18:58:56 dist-test-slave-6nmv krb5kdc[32195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561136, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:58:56.669711 504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:56.661597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.29.112.254:39707)
0812 18:58:56.661901 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:56.661908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:56.661923 (+ 15us) server_negotiation.cc:407] Connection header received
0812 18:58:56.662029 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:56.662031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:56.662076 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:56.662155 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:58:56.662830 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:56.663373 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:56.664207 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:56.664455 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:56.664545 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:56.666942 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:58:56.666946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:58:56.666948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:58:56.666970 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:58:56.668318 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:56.668689 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:56.668691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:56.668693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:56.668768 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:58:56.668995 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:58:56.668997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:58:56.668998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:58:56.669160 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:58:56.669259 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:56.669475 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 18:58:56.669589 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":109,"threads_started":1}
I20260812 18:58:56.672237 32248 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:58:56.672698 32248 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:58:56.677652 32426 tablet_service.cc:1467] Processing CreateTablet for tablet a25e620fe05c4f6aab14eb2dce0971e4 (DEFAULT_TABLE table=test-table [id=9909875f9bcd4ad2a3fe22fac868b6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:56.677757 32589 tablet_service.cc:1467] Processing CreateTablet for tablet a25e620fe05c4f6aab14eb2dce0971e4 (DEFAULT_TABLE table=test-table [id=9909875f9bcd4ad2a3fe22fac868b6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:56.678046 32426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a25e620fe05c4f6aab14eb2dce0971e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:56.678144 32589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a25e620fe05c4f6aab14eb2dce0971e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:56.680001 505 tablet_bootstrap.cc:492] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b: Bootstrap starting.
I20260812 18:58:56.680369 32756 tablet_service.cc:1467] Processing CreateTablet for tablet a25e620fe05c4f6aab14eb2dce0971e4 (DEFAULT_TABLE table=test-table [id=9909875f9bcd4ad2a3fe22fac868b6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:56.680495 506 tablet_bootstrap.cc:492] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf: Bootstrap starting.
I20260812 18:58:56.680668 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a25e620fe05c4f6aab14eb2dce0971e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:56.681128 505 tablet_bootstrap.cc:654] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:56.681383 506 tablet_bootstrap.cc:654] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:56.682555 506 tablet_bootstrap.cc:492] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf: No bootstrap required, opened a new log
I20260812 18:58:56.682652 506 ts_tablet_manager.cc:1397] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 18:58:56.683154 506 raft_consensus.cc:348] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:56.683246 510 tablet_bootstrap.cc:492] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0: Bootstrap starting.
I20260812 18:58:56.683272 506 raft_consensus.cc:374] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:56.683311 506 raft_consensus.cc:729] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4fec56450d4561b9d9c310f818f3cf, State: Initialized, Role: FOLLOWER
I20260812 18:58:56.683439 506 consensus_queue.cc:260] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:56.683722 506 ts_tablet_manager.cc:1428] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 18:58:56.684162 510 tablet_bootstrap.cc:654] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:56.684286 505 tablet_bootstrap.cc:492] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b: No bootstrap required, opened a new log
I20260812 18:58:56.684377 505 ts_tablet_manager.cc:1397] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260812 18:58:56.684823 505 raft_consensus.cc:348] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:56.684911 505 raft_consensus.cc:374] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:56.684957 505 raft_consensus.cc:729] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47444a585872484187527e66e2a51d5b, State: Initialized, Role: FOLLOWER
I20260812 18:58:56.685083 505 consensus_queue.cc:260] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:56.685511 505 ts_tablet_manager.cc:1428] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 18:58:56.686100 510 tablet_bootstrap.cc:492] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0: No bootstrap required, opened a new log
I20260812 18:58:56.686185 510 ts_tablet_manager.cc:1397] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 18:58:56.686609 510 raft_consensus.cc:348] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:56.686719 510 raft_consensus.cc:374] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:56.686755 510 raft_consensus.cc:729] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ad583b874b65a40dc2bb50caf4d0, State: Initialized, Role: FOLLOWER
I20260812 18:58:56.686888 510 consensus_queue.cc:260] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:56.687177 510 ts_tablet_manager.cc:1428] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 18:58:57.023422 512 raft_consensus.cc:482] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:58:57.023589 512 raft_consensus.cc:504] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.024806 512 leader_election.cc:288] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194:42649), 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195:33543)
I20260812 18:58:57.030434 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:57.025199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:43461 (local address 127.29.112.195:33543)
0812 18:58:57.025548 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:57.025558 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:57.025684 (+ 126us) server_negotiation.cc:407] Connection header received
0812 18:58:57.025945 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:57.025948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:57.025997 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:57.026094 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:57.026563 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.027393 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.029142 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.029742 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.029888 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:57.029955 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:57.030212 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 18:58:57.030280 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":123,"threads_started":1}
I20260812 18:58:57.030434 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:57.025202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:59585 (local address 127.29.112.194:42649)
0812 18:58:57.025548 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:57.025559 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:57.025684 (+ 125us) server_negotiation.cc:407] Connection header received
0812 18:58:57.025899 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:57.025905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:57.025982 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:57.026095 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:58:57.026563 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.027399 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.029142 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.029742 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.029888 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:57.029939 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:57.030212 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 18:58:57.030280 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":117,"threads_started":1}
I20260812 18:58:57.030509 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:57.025077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42649 (local address 127.29.112.193:59585)
0812 18:58:57.025564 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:57.025580 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:57.025727 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:57.026137 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:57.026140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:57.026169 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:57.026395 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:57.026401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.027519 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.027525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:57.029018 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:57.029027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.029904 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.029907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:57.030059 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:57.030113 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 18:58:57.030185 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 18:58:57.030301 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":316,"spinlock_wait_cycles":16256,"thread_start_us":159,"threads_started":1}
I20260812 18:58:57.030613 514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:57.025077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:33543 (local address 127.29.112.193:43461)
0812 18:58:57.025594 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:57.025610 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:57.025727 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:57.026137 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:57.026140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:57.026170 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:58:57.026393 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:57.026401 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.027539 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.027541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:57.029017 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:57.029027 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.029895 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.029899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:57.030062 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:57.030113 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 18:58:57.030185 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 18:58:57.030301 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":246,"spinlock_wait_cycles":28800,"thread_start_us":137,"threads_started":1}
I20260812 18:58:57.030997 32616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a25e620fe05c4f6aab14eb2dce0971e4" candidate_uuid: "47444a585872484187527e66e2a51d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4fec56450d4561b9d9c310f818f3cf" is_pre_election: true
I20260812 18:58:57.030999 310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a25e620fe05c4f6aab14eb2dce0971e4" candidate_uuid: "47444a585872484187527e66e2a51d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ad583b874b65a40dc2bb50caf4d0" is_pre_election: true
I20260812 18:58:57.031118 310 raft_consensus.cc:2454] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47444a585872484187527e66e2a51d5b in term 0.
I20260812 18:58:57.031150 32616 raft_consensus.cc:2454] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47444a585872484187527e66e2a51d5b in term 0.
I20260812 18:58:57.031636 32369 leader_election.cc:302] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0e4fec56450d4561b9d9c310f818f3cf, 47444a585872484187527e66e2a51d5b, 7616ad583b874b65a40dc2bb50caf4d0; no voters:
I20260812 18:58:57.031988 512 raft_consensus.cc:2781] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:58:57.032078 512 raft_consensus.cc:482] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:58:57.032133 512 raft_consensus.cc:3037] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:57.033391 512 raft_consensus.cc:504] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.033785 512 leader_election.cc:288] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [CANDIDATE]: Term 1 election: Requested vote from peers 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194:42649), 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195:33543)
I20260812 18:58:57.034183 32616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a25e620fe05c4f6aab14eb2dce0971e4" candidate_uuid: "47444a585872484187527e66e2a51d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4fec56450d4561b9d9c310f818f3cf"
I20260812 18:58:57.034323 32616 raft_consensus.cc:3037] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:57.034507 310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a25e620fe05c4f6aab14eb2dce0971e4" candidate_uuid: "47444a585872484187527e66e2a51d5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ad583b874b65a40dc2bb50caf4d0"
I20260812 18:58:57.034606 310 raft_consensus.cc:3037] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:57.035597 32616 raft_consensus.cc:2454] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47444a585872484187527e66e2a51d5b in term 1.
I20260812 18:58:57.035717 310 raft_consensus.cc:2454] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47444a585872484187527e66e2a51d5b in term 1.
I20260812 18:58:57.035909 32369 leader_election.cc:302] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4fec56450d4561b9d9c310f818f3cf, 47444a585872484187527e66e2a51d5b; no voters:
I20260812 18:58:57.036109 512 raft_consensus.cc:2781] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:57.036306 512 raft_consensus.cc:686] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 1 LEADER]: Becoming Leader. State: Replica: 47444a585872484187527e66e2a51d5b, State: Running, Role: LEADER
I20260812 18:58:57.036557 512 consensus_queue.cc:237] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.038764 32248 catalog_manager.cc:5495] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 47444a585872484187527e66e2a51d5b (127.29.112.193). New cstate: current_term: 1 leader_uuid: "47444a585872484187527e66e2a51d5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } health_report { overall_health: UNKNOWN } } }
I20260812 18:58:57.141624 530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:57.138294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59460 (local address 127.29.112.193:39051)
0812 18:58:57.138525 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:57.138532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:57.138543 (+ 11us) server_negotiation.cc:407] Connection header received
0812 18:58:57.138615 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:57.138617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:57.138676 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:57.138806 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:57.138976 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.139508 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.140276 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.140479 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.140549 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:57.140766 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:57.141214 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:57.141303 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:57.141379 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 18:58:57.141428 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":108,"threads_started":1}
I20260812 18:58:57.151232 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:57.148357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:42137 (local address 127.29.112.194:42649)
0812 18:58:57.148456 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:57.148459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:57.148471 (+ 12us) server_negotiation.cc:407] Connection header received
0812 18:58:57.148551 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:57.148553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:57.148589 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:57.148648 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:57.148851 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.149438 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.150281 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.150552 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.150633 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:57.150863 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:57.150936 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:57.150988 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:57.151085 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 18:58:57.151135 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 18:58:57.151242 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:57.148150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42649 (local address 127.29.112.193:42137)
0812 18:58:57.148389 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:57.148401 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:57.148477 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:57.148669 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:57.148671 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:57.148677 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:58:57.148764 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:57.148768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.149553 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.149555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:57.150176 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:57.150185 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.150647 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.150649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:57.150717 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:57.150755 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:58:57.151005 (+ 250us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:58:57.151010 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 18:58:57.151061 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 18:58:57.151126 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":53,"threads_started":1}
I20260812 18:58:57.157096 32616 raft_consensus.cc:1261] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Refusing update from remote peer 47444a585872484187527e66e2a51d5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:57.157212 310 raft_consensus.cc:1261] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Refusing update from remote peer 47444a585872484187527e66e2a51d5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:57.157600 512 consensus_queue.cc:1035] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:57.157805 517 consensus_queue.cc:1035] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:57.175823 530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:57.172269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:43335 (local address 127.29.112.193:39051)
0812 18:58:57.172372 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:57.172376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:57.172589 (+ 213us) server_negotiation.cc:407] Connection header received
0812 18:58:57.172663 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:57.172665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:57.172705 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:57.172766 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:57.173080 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.173679 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.174695 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.175046 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.175165 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:57.175406 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:57.175482 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:57.175561 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:57.175650 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 18:58:57.175700 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 18:58:57.175890 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:57.172227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:39051 (local address 127.29.112.194:43335)
0812 18:58:57.172488 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:57.172502 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:58:57.172599 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:58:57.172799 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:58:57.172800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:58:57.172806 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:58:57.172889 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:57.172925 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.173778 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.173780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:57.174588 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:58:57.174599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.175164 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.175167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:58:57.175262 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:57.175288 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:58:57.175568 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:58:57.175575 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 18:58:57.175641 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 18:58:57.175721 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":119,"threads_started":1}
I20260812 18:58:57.197088 32248 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:57.197247 32248 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:57.199304 32248 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff: Sending DeleteTablet for 3 replicas of tablet a25e620fe05c4f6aab14eb2dce0971e4
I20260812 18:58:57.199836 32426 tablet_service.cc:1514] Processing DeleteTablet for tablet a25e620fe05c4f6aab14eb2dce0971e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:57 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:59458
I20260812 18:58:57.199910 32756 tablet_service.cc:1514] Processing DeleteTablet for tablet a25e620fe05c4f6aab14eb2dce0971e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:57 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:39056
I20260812 18:58:57.200150 545 tablet_replica.cc:330] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0: stopping tablet replica
I20260812 18:58:57.200135 32589 tablet_service.cc:1514] Processing DeleteTablet for tablet a25e620fe05c4f6aab14eb2dce0971e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:57 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:58842
I20260812 18:58:57.200284 545 raft_consensus.cc:2229] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:57.200409 545 raft_consensus.cc:2258] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:57.200732 544 tablet_replica.cc:330] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b: stopping tablet replica
I20260812 18:58:57.200453 546 tablet_replica.cc:330] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf: stopping tablet replica
I20260812 18:58:57.200850 544 raft_consensus.cc:2229] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:58:57.201047 546 raft_consensus.cc:2229] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:57.201136 544 raft_consensus.cc:2258] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:57.201225 32248 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:58:57.201609 32248 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:58:57.201723 544 ts_tablet_manager.cc:1905] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:57.201887 546 raft_consensus.cc:2258] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:57.202149 545 ts_tablet_manager.cc:1905] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:57.202606 546 ts_tablet_manager.cc:1905] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:57.204488 544 ts_tablet_manager.cc:1918] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:58:57.204591 544 log.cc:1196] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-0/wal/wals/a25e620fe05c4f6aab14eb2dce0971e4
I20260812 18:58:57.204896 544 ts_tablet_manager.cc:1939] T a25e620fe05c4f6aab14eb2dce0971e4 P 47444a585872484187527e66e2a51d5b: Deleting consensus metadata
I20260812 18:58:57.205634 32236 catalog_manager.cc:4841] TS 47444a585872484187527e66e2a51d5b (127.29.112.193:39051): tablet a25e620fe05c4f6aab14eb2dce0971e4 (table test-table [id=9909875f9bcd4ad2a3fe22fac868b6eb]) successfully deleted
I20260812 18:58:57.207185 32426 tablet_service.cc:1467] Processing CreateTablet for tablet a36309cecdf0492b9e5c7bb655142270 (DEFAULT_TABLE table=test-table [id=ffb797f1d8aa4619a6faa5a77ad00cd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:57.207252 32756 tablet_service.cc:1467] Processing CreateTablet for tablet a36309cecdf0492b9e5c7bb655142270 (DEFAULT_TABLE table=test-table [id=ffb797f1d8aa4619a6faa5a77ad00cd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:57.207429 32426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a36309cecdf0492b9e5c7bb655142270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:57.207453 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a36309cecdf0492b9e5c7bb655142270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:57.208385 545 ts_tablet_manager.cc:1918] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:58:57.209564 546 ts_tablet_manager.cc:1918] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:58:57.209694 545 log.cc:1196] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/a25e620fe05c4f6aab14eb2dce0971e4
I20260812 18:58:57.210042 545 ts_tablet_manager.cc:1939] T a25e620fe05c4f6aab14eb2dce0971e4 P 7616ad583b874b65a40dc2bb50caf4d0: Deleting consensus metadata
I20260812 18:58:57.210390 548 tablet_bootstrap.cc:492] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b: Bootstrap starting.
I20260812 18:58:57.210074 32589 tablet_service.cc:1467] Processing CreateTablet for tablet a36309cecdf0492b9e5c7bb655142270 (DEFAULT_TABLE table=test-table [id=ffb797f1d8aa4619a6faa5a77ad00cd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:58:57.210670 32589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a36309cecdf0492b9e5c7bb655142270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:58:57.211416 548 tablet_bootstrap.cc:654] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:57.212127 547 tablet_bootstrap.cc:492] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0: Bootstrap starting.
I20260812 18:58:57.212569 548 tablet_bootstrap.cc:492] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b: No bootstrap required, opened a new log
I20260812 18:58:57.212643 548 ts_tablet_manager.cc:1397] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 18:58:57.213076 546 log.cc:1196] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/a25e620fe05c4f6aab14eb2dce0971e4
I20260812 18:58:57.213317 551 tablet_bootstrap.cc:492] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf: Bootstrap starting.
I20260812 18:58:57.213260 548 raft_consensus.cc:348] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.213366 548 raft_consensus.cc:374] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:57.213416 548 raft_consensus.cc:729] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47444a585872484187527e66e2a51d5b, State: Initialized, Role: FOLLOWER
I20260812 18:58:57.213547 548 consensus_queue.cc:260] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.213761 546 ts_tablet_manager.cc:1939] T a25e620fe05c4f6aab14eb2dce0971e4 P 0e4fec56450d4561b9d9c310f818f3cf: Deleting consensus metadata
I20260812 18:58:57.214011 32233 catalog_manager.cc:4841] TS 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195:33543): tablet a25e620fe05c4f6aab14eb2dce0971e4 (table test-table [id=9909875f9bcd4ad2a3fe22fac868b6eb]) successfully deleted
I20260812 18:58:57.214435 551 tablet_bootstrap.cc:654] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:57.214564 32234 catalog_manager.cc:4841] TS 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194:42649): tablet a25e620fe05c4f6aab14eb2dce0971e4 (table test-table [id=9909875f9bcd4ad2a3fe22fac868b6eb]) successfully deleted
I20260812 18:58:57.215220 548 ts_tablet_manager.cc:1428] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260812 18:58:57.216210 547 tablet_bootstrap.cc:654] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0: Neither blocks nor log segments found. Creating new log.
I20260812 18:58:57.217384 547 tablet_bootstrap.cc:492] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0: No bootstrap required, opened a new log
I20260812 18:58:57.217479 547 ts_tablet_manager.cc:1397] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 18:58:57.217501 551 tablet_bootstrap.cc:492] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf: No bootstrap required, opened a new log
I20260812 18:58:57.217585 551 ts_tablet_manager.cc:1397] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 18:58:57.217864 547 raft_consensus.cc:348] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.217978 547 raft_consensus.cc:374] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:57.217949 551 raft_consensus.cc:348] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.218022 547 raft_consensus.cc:729] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ad583b874b65a40dc2bb50caf4d0, State: Initialized, Role: FOLLOWER
I20260812 18:58:57.218050 551 raft_consensus.cc:374] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:58:57.218081 551 raft_consensus.cc:729] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4fec56450d4561b9d9c310f818f3cf, State: Initialized, Role: FOLLOWER
I20260812 18:58:57.218148 547 consensus_queue.cc:260] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.218182 551 consensus_queue.cc:260] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.218454 551 ts_tablet_manager.cc:1428] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:58:57.218887 547 ts_tablet_manager.cc:1428] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:58:57.221899 479 raft_consensus.cc:482] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:58:57.222015 479 raft_consensus.cc:504] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.222364 479 leader_election.cc:288] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47444a585872484187527e66e2a51d5b (127.29.112.193:39051), 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194:42649)
I20260812 18:58:57.222632 32451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36309cecdf0492b9e5c7bb655142270" candidate_uuid: "7616ad583b874b65a40dc2bb50caf4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47444a585872484187527e66e2a51d5b" is_pre_election: true
I20260812 18:58:57.222756 32451 raft_consensus.cc:2454] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ad583b874b65a40dc2bb50caf4d0 in term 0.
I20260812 18:58:57.222744 32616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36309cecdf0492b9e5c7bb655142270" candidate_uuid: "7616ad583b874b65a40dc2bb50caf4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4fec56450d4561b9d9c310f818f3cf" is_pre_election: true
I20260812 18:58:57.222851 32616 raft_consensus.cc:2454] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ad583b874b65a40dc2bb50caf4d0 in term 0.
I20260812 18:58:57.223026 32703 leader_election.cc:302] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47444a585872484187527e66e2a51d5b, 7616ad583b874b65a40dc2bb50caf4d0; no voters:
I20260812 18:58:57.223201 479 raft_consensus.cc:2781] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:58:57.223259 479 raft_consensus.cc:482] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:58:57.223312 479 raft_consensus.cc:3037] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:57.224303 479 raft_consensus.cc:504] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.224644 479 leader_election.cc:288] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [CANDIDATE]: Term 1 election: Requested vote from peers 47444a585872484187527e66e2a51d5b (127.29.112.193:39051), 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194:42649)
I20260812 18:58:57.224975 32451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36309cecdf0492b9e5c7bb655142270" candidate_uuid: "7616ad583b874b65a40dc2bb50caf4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47444a585872484187527e66e2a51d5b"
I20260812 18:58:57.224995 32616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36309cecdf0492b9e5c7bb655142270" candidate_uuid: "7616ad583b874b65a40dc2bb50caf4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4fec56450d4561b9d9c310f818f3cf"
I20260812 18:58:57.225090 32616 raft_consensus.cc:3037] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:57.225090 32451 raft_consensus.cc:3037] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:58:57.226191 32451 raft_consensus.cc:2454] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ad583b874b65a40dc2bb50caf4d0 in term 1.
I20260812 18:58:57.226191 32616 raft_consensus.cc:2454] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ad583b874b65a40dc2bb50caf4d0 in term 1.
I20260812 18:58:57.226444 32701 leader_election.cc:302] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4fec56450d4561b9d9c310f818f3cf, 7616ad583b874b65a40dc2bb50caf4d0; no voters:
I20260812 18:58:57.226706 479 raft_consensus.cc:2781] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:58:57.226836 479 raft_consensus.cc:686] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 LEADER]: Becoming Leader. State: Replica: 7616ad583b874b65a40dc2bb50caf4d0, State: Running, Role: LEADER
I20260812 18:58:57.226945 479 consensus_queue.cc:237] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } }
I20260812 18:58:57.228333 32249 catalog_manager.cc:5495] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195). New cstate: current_term: 1 leader_uuid: "7616ad583b874b65a40dc2bb50caf4d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7616ad583b874b65a40dc2bb50caf4d0" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33543 } health_report { overall_health: HEALTHY } } }
I20260812 18:58:57.246730 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:57.243657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39088 (local address 127.29.112.195:33543)
0812 18:58:57.243761 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 18:58:57.243766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:58:57.243804 (+ 38us) server_negotiation.cc:407] Connection header received
0812 18:58:57.243962 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:58:57.243965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:58:57.244012 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:58:57.244130 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:58:57.244372 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.244885 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.245672 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:58:57.245961 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:58:57.246060 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:58:57.246316 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:58:57.246385 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:58:57.246475 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 18:58:57.246561 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 18:58:57.246608 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 18:58:57.252624 32616 raft_consensus.cc:1261] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Refusing update from remote peer 7616ad583b874b65a40dc2bb50caf4d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:57.252975 32451 raft_consensus.cc:1261] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Refusing update from remote peer 7616ad583b874b65a40dc2bb50caf4d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:58:57.253029 479 consensus_queue.cc:1035] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e4fec56450d4561b9d9c310f818f3cf" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:58:57.253351 479 consensus_queue.cc:1035] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47444a585872484187527e66e2a51d5b" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 39051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 18:58:57.267805 524 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 18:58:58.819638 32248 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:58.819808 32248 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:58:58.822162 32248 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad288562d7274a28a73731e961d362ff: Sending DeleteTablet for 3 replicas of tablet a36309cecdf0492b9e5c7bb655142270
I20260812 18:58:58.822686 32426 tablet_service.cc:1514] Processing DeleteTablet for tablet a36309cecdf0492b9e5c7bb655142270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:58 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:59458
I20260812 18:58:58.822702 32589 tablet_service.cc:1514] Processing DeleteTablet for tablet a36309cecdf0492b9e5c7bb655142270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:58 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:58842
I20260812 18:58:58.822890 32756 tablet_service.cc:1514] Processing DeleteTablet for tablet a36309cecdf0492b9e5c7bb655142270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:58:58 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:39056
I20260812 18:58:58.823060 585 tablet_replica.cc:330] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf: stopping tablet replica
I20260812 18:58:58.823060 586 tablet_replica.cc:330] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b: stopping tablet replica
I20260812 18:58:58.823186 585 raft_consensus.cc:2229] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:58.823186 586 raft_consensus.cc:2229] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:58:58.823294 585 raft_consensus.cc:2258] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:58.823294 586 raft_consensus.cc:2258] T a36309cecdf0492b9e5c7bb655142270 P 47444a585872484187527e66e2a51d5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:58.823552 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 32344
I20260812 18:58:58.826957 587 tablet_replica.cc:330] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0: stopping tablet replica
I20260812 18:58:58.827158 587 raft_consensus.cc:2229] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:58:58.827353 587 raft_consensus.cc:2258] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:58:58.827673 585 ts_tablet_manager.cc:1905] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:58.827891 587 ts_tablet_manager.cc:1905] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:58:58.830446 587 ts_tablet_manager.cc:1918] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 18:58:58.830446 585 ts_tablet_manager.cc:1918] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 18:58:58.830528 587 log.cc:1196] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/a36309cecdf0492b9e5c7bb655142270
I20260812 18:58:58.830538 585 log.cc:1196] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/a36309cecdf0492b9e5c7bb655142270
I20260812 18:58:58.830814 585 ts_tablet_manager.cc:1939] T a36309cecdf0492b9e5c7bb655142270 P 0e4fec56450d4561b9d9c310f818f3cf: Deleting consensus metadata
I20260812 18:58:58.830814 587 ts_tablet_manager.cc:1939] T a36309cecdf0492b9e5c7bb655142270 P 7616ad583b874b65a40dc2bb50caf4d0: Deleting consensus metadata
I20260812 18:58:58.831429 32233 catalog_manager.cc:4841] TS 7616ad583b874b65a40dc2bb50caf4d0 (127.29.112.195:33543): tablet a36309cecdf0492b9e5c7bb655142270 (table test-table [id=ffb797f1d8aa4619a6faa5a77ad00cd9]) successfully deleted
I20260812 18:58:58.831504 32234 catalog_manager.cc:4841] TS 0e4fec56450d4561b9d9c310f818f3cf (127.29.112.194:42649): tablet a36309cecdf0492b9e5c7bb655142270 (table test-table [id=ffb797f1d8aa4619a6faa5a77ad00cd9]) successfully deleted
I20260812 18:58:58.835073 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 32518
W20260812 18:58:58.835104 32236 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:39051 (error 108)
I20260812 18:58:58.838496 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:58:58.838127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:39051 (local address 127.0.0.1:59468)
0812 18:58:58.838352 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 18:58:58.838419 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
W20260812 18:58:58.838702 32236 catalog_manager.cc:4584] TS 47444a585872484187527e66e2a51d5b (127.29.112.193:39051): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a36309cecdf0492b9e5c7bb655142270: Network error: Client connection negotiation failed: client connection to 127.29.112.193:39051: connect: Connection refused (error 111)
I20260812 18:58:58.844605 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 32681
I20260812 18:58:58.853425 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 32214
2026-08-12T18:58:58Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6422 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:58:58 dist-test-slave-6nmv krb5kdc[592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:58:58 dist-test-slave-6nmv krb5kdc[592](info): set up 2 sockets
Aug 12 18:58:58 dist-test-slave-6nmv krb5kdc[592](info): commencing operation
krb5kdc: starting...
W20260812 18:58:59.932804 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:00 dist-test-slave-6nmv krb5kdc[592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:59:00.053268 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:44757
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:44757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:00.163657 606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:00.163894 606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:00.163933 606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:00.167748 606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:00.167793 606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:00.167814 606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:00.167829 606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:00.167845 606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:00.172802 606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:44757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:44757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:00.173705 606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:00.174636 606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:00.183053 611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:00.183063 612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:00.183091 606 server_base.cc:1034] running on GCE node
W20260812 18:59:00.183058 616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:00.183735 606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:00.184752 606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:00.185922 606 hybrid_clock.cc:648] HybridClock initialized: now 1786561140185891 us; error 43 us; skew 500 ppm
Aug 12 18:59:00 dist-test-slave-6nmv krb5kdc[592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:00.188532 606 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:59:00.189388 606 webserver.cc:460] Webserver started at http://127.29.112.254:40729/ using document root <none> and password file <none>
I20260812 18:59:00.189996 606 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:00.190086 606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:00.190294 606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:00.192222 606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "19d877d54e264edab06a52ebafad192b"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "138428e07555f267b86111c5d700ef98"
server_key_iv: "2d2458e96af403c41b86111db08d7578"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.192780 606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "19d877d54e264edab06a52ebafad192b"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "138428e07555f267b86111c5d700ef98"
server_key_iv: "2d2458e96af403c41b86111db08d7578"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.196292 606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 18:59:00.198522 624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:00.199666 606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:00.199813 606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "19d877d54e264edab06a52ebafad192b"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "138428e07555f267b86111c5d700ef98"
server_key_iv: "2d2458e96af403c41b86111db08d7578"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.199935 606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:00.218010 606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:00.220495 606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:00.220690 606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:00.227170 606 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:44757
I20260812 18:59:00.227192 683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:44757 every 8 connection(s)
I20260812 18:59:00.227965 606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:00.229228 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 606
I20260812 18:59:00.229310 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:00.229552 30147 external_mini_cluster.cc:1428] Setting key 39ae02ca5f7fd84d924b3beffd2ac5b2
I20260812 18:59:00.230502 684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:00.235477 684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b: Bootstrap starting.
Aug 12 18:59:00 dist-test-slave-6nmv krb5kdc[592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:00.237902 684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:00.238637 684 log.cc:824] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:00.240582 684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b: No bootstrap required, opened a new log
I20260812 18:59:00.242425 687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:00.230813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39092 (local address 127.29.112.254:44757)
0812 18:59:00.231224 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:00.231237 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:00.231270 (+ 33us) server_negotiation.cc:407] Connection header received
0812 18:59:00.231885 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:00.231900 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:00.232189 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:00.232506 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:00.233248 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.234145 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.234979 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.235429 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.235613 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:00.238141 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:00.238146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:00.238157 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:00.238180 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:00.239835 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:00.240286 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:00.240290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:00.240294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:00.240391 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:00.240686 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:00.240689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:00.240691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:00.241033 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:00.241172 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:00.241437 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 18:59:00.241727 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":137,"threads_started":1}
I20260812 18:59:00.244583 684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d877d54e264edab06a52ebafad192b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44757 } }
I20260812 18:59:00.244820 684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:00.244899 684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19d877d54e264edab06a52ebafad192b, State: Initialized, Role: FOLLOWER
I20260812 18:59:00.245307 684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d877d54e264edab06a52ebafad192b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44757 } }
I20260812 18:59:00.245445 684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:00.245507 684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:00.245616 684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:00.246687 684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d877d54e264edab06a52ebafad192b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44757 } }
I20260812 18:59:00.247001 684 leader_election.cc:302] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19d877d54e264edab06a52ebafad192b; no voters:
I20260812 18:59:00.247334 684 leader_election.cc:288] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:00.247442 689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:00.247710 689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [term 1 LEADER]: Becoming Leader. State: Replica: 19d877d54e264edab06a52ebafad192b, State: Running, Role: LEADER
I20260812 18:59:00.248036 689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d877d54e264edab06a52ebafad192b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44757 } }
I20260812 18:59:00.248473 684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:00.249583 691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19d877d54e264edab06a52ebafad192b. Latest consensus state: current_term: 1 leader_uuid: "19d877d54e264edab06a52ebafad192b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d877d54e264edab06a52ebafad192b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44757 } } }
I20260812 18:59:00.249711 691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:00.249974 690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19d877d54e264edab06a52ebafad192b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d877d54e264edab06a52ebafad192b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44757 } } }
I20260812 18:59:00.250130 690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:00.250722 699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:00.253376 699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:00.258018 699 catalog_manager.cc:1347] Generated new cluster ID: 89b93759e0fe45b59b61cec5673a6cdc
I20260812 18:59:00.258080 699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:00.279279 699 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:00.280277 699 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:00.285786 699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19d877d54e264edab06a52ebafad192b: Generated new TSK 0
I20260812 18:59:00.286576 699 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 18:59:00.322049 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44757
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:00.426662 716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:00.426983 716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:00.427062 716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:00.430935 716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:00.431008 716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:00.431087 716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:00.435880 716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:00.436738 716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:00.437608 716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:00.446255 725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:00.446265 722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:00.446255 723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:00.446472 716 server_base.cc:1034] running on GCE node
I20260812 18:59:00.446920 716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:00.447428 716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:00.448626 716 hybrid_clock.cc:648] HybridClock initialized: now 1786561140448592 us; error 44 us; skew 500 ppm
Aug 12 18:59:00 dist-test-slave-6nmv krb5kdc[592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:00.451342 716 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 18:59:00.452126 716 webserver.cc:460] Webserver started at http://127.29.112.193:44831/ using document root <none> and password file <none>
I20260812 18:59:00.452647 716 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:00.452734 716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:00.452944 716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:00.454885 716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "fc7ba5a487424266922609d2243b33bc"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "b2ce2050690e7acad275dfd874fcb956"
server_key_iv: "440fbebbd22b20436204a1058cd7c5f3"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.455439 716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "fc7ba5a487424266922609d2243b33bc"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "b2ce2050690e7acad275dfd874fcb956"
server_key_iv: "440fbebbd22b20436204a1058cd7c5f3"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.458763 716 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 18:59:00.460801 733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:00.461851 716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:00.461979 716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "fc7ba5a487424266922609d2243b33bc"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "b2ce2050690e7acad275dfd874fcb956"
server_key_iv: "440fbebbd22b20436204a1058cd7c5f3"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.462077 716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:00.479050 716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:00.482007 716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:00.482182 716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:00.482703 716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:00.483434 716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:00.483502 716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:00.483565 716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:00.483606 716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:00.492745 716 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:35827
I20260812 18:59:00.492791 874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:35827 every 8 connection(s)
I20260812 18:59:00.493556 716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:00.497535 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 716
I20260812 18:59:00.497646 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:00.498041 30147 external_mini_cluster.cc:1428] Setting key 98e40a7a432450e0f85ff5f25ed6937c
Aug 12 18:59:00 dist-test-slave-6nmv krb5kdc[592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:00.508486 687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:00.495281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:55333 (local address 127.29.112.254:44757)
0812 18:59:00.495414 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:00.495418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:00.496163 (+ 745us) server_negotiation.cc:407] Connection header received
0812 18:59:00.497754 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:00.497761 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:00.497844 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:00.497928 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:00.499202 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.499812 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.501551 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.501949 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.502061 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:00.504822 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:00.504826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:00.504829 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:00.504859 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:00.506561 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:00.507012 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:00.507016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:00.507019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:00.507111 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:00.507406 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:00.507409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:00.507412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:00.507647 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:00.507737 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:00.508165 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 18:59:00.508311 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 18:59:00.509187 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:00.495631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44757 (local address 127.29.112.193:55333)
0812 18:59:00.496024 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:00.496064 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:00.497509 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:00.498133 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:00.498144 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:00.498593 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:00.499043 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:00.499073 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.499969 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.499975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:00.501401 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:00.501429 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.502081 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.502084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:00.502235 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:00.502859 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:00.502884 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:00.504662 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:00.506674 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:00.506678 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:00.506688 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:00.506914 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:00.507202 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:00.507204 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:00.507205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:00.507319 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:00.507750 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:00.507754 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:00.507944 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 18:59:00.508084 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 18:59:00.508304 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":104,"threads_started":1}
I20260812 18:59:00.510337 876 heartbeater.cc:347] Connected to a master server at 127.29.112.254:44757
I20260812 18:59:00.510545 876 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:00.511013 876 heartbeater.cc:511] Master 127.29.112.254:44757 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
I20260812 18:59:00.512535 645 ts_manager.cc:194] Registered new tserver with Master: fc7ba5a487424266922609d2243b33bc (127.29.112.193:35827)
I20260812 18:59:00.514045 645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:55333
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 18:59:00.527020 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44757
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:00.638449 883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:00.638670 883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:00.638710 883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:00.642606 883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:00.642675 883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:00.642781 883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:00.647692 883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:00.648617 883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:00.649609 883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:00.658356 892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:00.658396 890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:00.658455 889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:00.658444 883 server_base.cc:1034] running on GCE node
I20260812 18:59:00.659058 883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:00.659567 883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:00.660737 883 hybrid_clock.cc:648] HybridClock initialized: now 1786561140660710 us; error 40 us; skew 500 ppm
Aug 12 18:59:00 dist-test-slave-6nmv krb5kdc[592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:00.663522 883 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 18:59:00.664268 883 webserver.cc:460] Webserver started at http://127.29.112.194:43103/ using document root <none> and password file <none>
I20260812 18:59:00.664763 883 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:00.664821 883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:00.664973 883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:00.666924 883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "c4c612c700d94faba1ace25358d62442"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "64f95efa7d1e27b178bec3432ef35248"
server_key_iv: "e1a8183a781b5685c55dbd2afd22f8ff"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.667372 883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "c4c612c700d94faba1ace25358d62442"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "64f95efa7d1e27b178bec3432ef35248"
server_key_iv: "e1a8183a781b5685c55dbd2afd22f8ff"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.670624 883 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 18:59:00.673020 900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:00.674328 883 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:00.674431 883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "c4c612c700d94faba1ace25358d62442"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "64f95efa7d1e27b178bec3432ef35248"
server_key_iv: "e1a8183a781b5685c55dbd2afd22f8ff"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.674561 883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:00.697147 883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:00.700385 883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:00.700590 883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:00.701166 883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:00.701980 883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:00.702028 883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:00.702064 883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:00.702077 883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:00.711170 883 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:42661
I20260812 18:59:00.711201 1030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:42661 every 8 connection(s)
I20260812 18:59:00.712005 883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:00.713932 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 883
I20260812 18:59:00.714035 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:00.714375 30147 external_mini_cluster.cc:1428] Setting key 4ed374d057340d9b5294e96904d97862
Aug 12 18:59:00 dist-test-slave-6nmv krb5kdc[592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
I20260812 18:59:00.725713 687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:00.713693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:47475 (local address 127.29.112.254:44757)
0812 18:59:00.713882 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:00.713891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:00.714886 (+ 995us) server_negotiation.cc:407] Connection header received
0812 18:59:00.716338 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:00.716340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:00.716401 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:00.716494 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:00.717525 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.718075 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.719156 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.719375 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.719490 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:00.722003 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:00.722006 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:00.722008 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:00.722029 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:00.723677 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:00.724157 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:00.724160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:00.724162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:00.724256 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:00.724614 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:00.724617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:00.724618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:00.724790 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:00.724944 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:00.725481 (+ 537us) server_negotiation.cc:299] Negotiation successful
0812 18:59:00.725578 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
I20260812 18:59:00.726765 1035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:00.714211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44757 (local address 127.29.112.194:47475)
0812 18:59:00.714709 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:00.714749 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:00.716145 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:00.716647 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:00.716654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:00.716971 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:00.717376 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:00.717388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.718205 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.718213 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:00.719032 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:00.719040 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.719523 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.719526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:00.719691 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:00.720240 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:00.720255 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:00.721851 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:00.723784 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:00.723788 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:00.723798 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:00.724080 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:00.724346 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:00.724348 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:00.724349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:00.724512 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:00.724987 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:00.724997 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:00.725250 (+ 253us) client_negotiation.cc:769] Sending connection context
0812 18:59:00.725435 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 18:59:00.725685 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":152,"threads_started":1}
I20260812 18:59:00.728000 1033 heartbeater.cc:347] Connected to a master server at 127.29.112.254:44757
I20260812 18:59:00.728229 1033 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:00.728760 1033 heartbeater.cc:511] Master 127.29.112.254:44757 requested a full tablet report, sending...
I20260812 18:59:00.729977 645 ts_manager.cc:194] Registered new tserver with Master: c4c612c700d94faba1ace25358d62442 (127.29.112.194:42661)
I20260812 18:59:00.730588 645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:47475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 18:59:00.742468 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44757
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:00.851778 1039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:00.852085 1039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:00.852133 1039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:00.856565 1039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:00.856629 1039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:00.856725 1039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:00.861646 1039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:00.862639 1039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:00.863587 1039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:00.872699 1047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:00.872692 1044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:00.872692 1045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:00.873046 1039 server_base.cc:1034] running on GCE node
I20260812 18:59:00.873411 1039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:00.874066 1039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:00.875252 1039 hybrid_clock.cc:648] HybridClock initialized: now 1786561140875240 us; error 26 us; skew 500 ppm
Aug 12 18:59:00 dist-test-slave-6nmv krb5kdc[592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:00.877874 1039 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 18:59:00.878773 1039 webserver.cc:460] Webserver started at http://127.29.112.195:36549/ using document root <none> and password file <none>
I20260812 18:59:00.879292 1039 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:00.879377 1039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:00.879591 1039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:00.881603 1039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "9b91234255094e8b99c9d98b46350a47"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "1013bf490ed64ae78731de07120cd662"
server_key_iv: "ad7e8850a5fc87c8095ed7de8f0bd8d7"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.882223 1039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "9b91234255094e8b99c9d98b46350a47"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "1013bf490ed64ae78731de07120cd662"
server_key_iv: "ad7e8850a5fc87c8095ed7de8f0bd8d7"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.885583 1039 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 18:59:00.887887 1057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:00.888983 1039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:00.889098 1039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "9b91234255094e8b99c9d98b46350a47"
format_stamp: "Formatted at 2026-08-12 18:59:00 on dist-test-slave-6nmv"
server_key: "1013bf490ed64ae78731de07120cd662"
server_key_iv: "ad7e8850a5fc87c8095ed7de8f0bd8d7"
server_key_version: "encryptionkey@0"
I20260812 18:59:00.889190 1039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:00.902405 1039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:00.905118 1039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:00.905314 1039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:00.905925 1039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:00.906661 1039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:00.906723 1039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:00.906783 1039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:00.906828 1039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:00.916087 1039 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:41143
I20260812 18:59:00.916126 1191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:41143 every 8 connection(s)
I20260812 18:59:00.916923 1039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:59:00.918335 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 1039
I20260812 18:59:00.918462 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:00.918838 30147 external_mini_cluster.cc:1428] Setting key 3a39956324fc60cdad1bf42d3826fc48
Aug 12 18:59:00 dist-test-slave-6nmv krb5kdc[592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:00.929993 687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:00.918901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:57595 (local address 127.29.112.254:44757)
0812 18:59:00.919051 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:00.919054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:00.920243 (+ 1189us) server_negotiation.cc:407] Connection header received
0812 18:59:00.921676 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:00.921679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:00.921728 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:00.921799 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:00.922854 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.923337 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.924287 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.924491 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.924596 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:00.926955 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:00.926957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:00.926959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:00.926983 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:00.928136 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:00.928546 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:00.928548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:00.928549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:00.928617 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:00.928954 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:00.928957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:00.928958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:00.929116 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:00.929190 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:00.929624 (+ 434us) server_negotiation.cc:299] Negotiation successful
0812 18:59:00.929719 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 18:59:00.930662 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:00.919413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44757 (local address 127.29.112.195:57595)
0812 18:59:00.920083 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:00.920129 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:00.921494 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:00.922006 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:00.922012 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:00.922326 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:00.922723 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:00.922734 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.923442 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.923444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:00.924181 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:00.924189 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:00.924640 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:00.924642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:00.924772 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:00.925318 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:00.925334 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:00.926794 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:00.928226 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:00.928230 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:00.928239 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:00.928464 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:00.928717 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:00.928720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:00.928721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:00.928852 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:00.929202 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:00.929207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:00.929426 (+ 219us) client_negotiation.cc:769] Sending connection context
0812 18:59:00.929581 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 18:59:00.929801 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":148,"threads_started":1}
I20260812 18:59:00.931788 1192 heartbeater.cc:347] Connected to a master server at 127.29.112.254:44757
I20260812 18:59:00.932050 1192 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:00.932644 1192 heartbeater.cc:511] Master 127.29.112.254:44757 requested a full tablet report, sending...
I20260812 18:59:00.933629 645 ts_manager.cc:194] Registered new tserver with Master: 9b91234255094e8b99c9d98b46350a47 (127.29.112.195:41143)
I20260812 18:59:00.934198 645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:57595
I20260812 18:59:00.944521 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 18:59:00.959923 687 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:00.958485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39098 (local address 127.29.112.254:44757)
0812 18:59:00.958680 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:00.958684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:00.958761 (+ 77us) server_negotiation.cc:407] Connection header received
0812 18:59:00.958878 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:00.958881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:00.958931 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:00.959000 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:00.959771 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39098: BlockingRecv error: recv got EOF from 127.0.0.1:39098 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 18:59:00.961186 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 716
I20260812 18:59:00.972062 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 883
I20260812 18:59:00.982139 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 1039
I20260812 18:59:00.990726 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 606
2026-08-12T18:59:00Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2134 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:59:01 dist-test-slave-6nmv krb5kdc[1204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:59:01 dist-test-slave-6nmv krb5kdc[1204](info): set up 2 sockets
Aug 12 18:59:01 dist-test-slave-6nmv krb5kdc[1204](info): commencing operation
krb5kdc: starting...
W20260812 18:59:02.095119 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:02 dist-test-slave-6nmv krb5kdc[1204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:59:02.217022 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:36905
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:34933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:36905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:02.322583 1217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:02.322867 1217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:02.322922 1217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:02.326872 1217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:02.326952 1217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:02.326990 1217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:02.327049 1217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:02.327087 1217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:02.331964 1217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:36905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:02.332899 1217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:02.333863 1217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:02.341837 1224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:02.341869 1223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:02.341835 1226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:02.342101 1217 server_base.cc:1034] running on GCE node
I20260812 18:59:02.342643 1217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:02.343652 1217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:02.344810 1217 hybrid_clock.cc:648] HybridClock initialized: now 1786561142344794 us; error 30 us; skew 500 ppm
Aug 12 18:59:02 dist-test-slave-6nmv krb5kdc[1204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561142, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:02.347543 1217 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:59:02.348408 1217 webserver.cc:460] Webserver started at http://127.29.112.254:37361/ using document root <none> and password file <none>
I20260812 18:59:02.348958 1217 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:02.349017 1217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:02.349247 1217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:02.351212 1217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "2744ff5a65024c838924b969dbdee6d8"
format_stamp: "Formatted at 2026-08-12 18:59:02 on dist-test-slave-6nmv"
server_key: "820bb68e525e52acb6b5343073763e51"
server_key_iv: "ecabd9256517bf73e57809a156cedd74"
server_key_version: "encryptionkey@0"
I20260812 18:59:02.351779 1217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "2744ff5a65024c838924b969dbdee6d8"
format_stamp: "Formatted at 2026-08-12 18:59:02 on dist-test-slave-6nmv"
server_key: "820bb68e525e52acb6b5343073763e51"
server_key_iv: "ecabd9256517bf73e57809a156cedd74"
server_key_version: "encryptionkey@0"
I20260812 18:59:02.355266 1217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260812 18:59:02.357561 1234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:02.358748 1217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 18:59:02.358841 1217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "2744ff5a65024c838924b969dbdee6d8"
format_stamp: "Formatted at 2026-08-12 18:59:02 on dist-test-slave-6nmv"
server_key: "820bb68e525e52acb6b5343073763e51"
server_key_iv: "ecabd9256517bf73e57809a156cedd74"
server_key_version: "encryptionkey@0"
I20260812 18:59:02.358933 1217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:02.377671 1217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:02.380834 1217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:02.381039 1217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:02.388289 1217 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:36905
I20260812 18:59:02.388669 1297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:36905 every 8 connection(s)
I20260812 18:59:02.389237 1217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:02.392215 1299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:02.392251 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 1217
I20260812 18:59:02.392405 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:02.392692 30147 external_mini_cluster.cc:1428] Setting key a8219ca4787478869c9f1e1a595c147b
I20260812 18:59:02.399158 1299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8: Bootstrap starting.
Aug 12 18:59:02 dist-test-slave-6nmv krb5kdc[1204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:02.401926 1299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:02.402791 1299 log.cc:824] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:02.404696 1299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8: No bootstrap required, opened a new log
I20260812 18:59:02.406622 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:02.394009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43668 (local address 127.29.112.254:36905)
0812 18:59:02.394416 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:02.394430 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:02.394464 (+ 34us) server_negotiation.cc:407] Connection header received
0812 18:59:02.395063 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:02.395079 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:02.395376 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:02.395640 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:02.396616 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:02.397538 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:02.398695 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:02.399133 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:02.399248 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:02.401991 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:02.401996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:02.402008 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:02.402059 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:02.403743 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:02.404194 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:02.404200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:02.404206 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:02.404339 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:02.404644 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:02.404647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:02.404648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:02.405088 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:02.405292 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:02.405584 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 18:59:02.405940 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":124,"threads_started":1}
I20260812 18:59:02.408591 1299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2744ff5a65024c838924b969dbdee6d8" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36905 } }
I20260812 18:59:02.408818 1299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:02.408902 1299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2744ff5a65024c838924b969dbdee6d8, State: Initialized, Role: FOLLOWER
I20260812 18:59:02.409358 1299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2744ff5a65024c838924b969dbdee6d8" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36905 } }
I20260812 18:59:02.409533 1299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:02.409629 1299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:02.409735 1299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:02.409860 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 18:59:02.410998 1299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2744ff5a65024c838924b969dbdee6d8" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36905 } }
I20260812 18:59:02.411366 1299 leader_election.cc:302] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2744ff5a65024c838924b969dbdee6d8; no voters:
I20260812 18:59:02.411759 1299 leader_election.cc:288] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:02.411924 1304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:02.412189 1304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [term 1 LEADER]: Becoming Leader. State: Replica: 2744ff5a65024c838924b969dbdee6d8, State: Running, Role: LEADER
I20260812 18:59:02.412500 1304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2744ff5a65024c838924b969dbdee6d8" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36905 } }
I20260812 18:59:02.413148 1299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:02.416271 1306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2744ff5a65024c838924b969dbdee6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2744ff5a65024c838924b969dbdee6d8" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36905 } } }
I20260812 18:59:02.416417 1306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:02.416730 1305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2744ff5a65024c838924b969dbdee6d8. Latest consensus state: current_term: 1 leader_uuid: "2744ff5a65024c838924b969dbdee6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2744ff5a65024c838924b969dbdee6d8" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36905 } } }
I20260812 18:59:02.416823 1305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8 [sys.catalog]: This master's current role is: LEADER
W20260812 18:59:02.418670 1324 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 18:59:02.418767 1324 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 18:59:02.418859 1325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:02.419541 1325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:02.424543 1325 catalog_manager.cc:1347] Generated new cluster ID: 19a8bdf8f8b449cca193bd4660146369
I20260812 18:59:02.424619 1325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:02.430751 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:02.425800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.29.112.254:36905)
0812 18:59:02.425958 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:02.425964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:02.426223 (+ 259us) server_negotiation.cc:407] Connection header received
0812 18:59:02.426390 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:02.426393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:02.426449 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:02.426544 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:02.427093 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:02.427634 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:02.428494 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:02.428760 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:02.428874 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:02.429752 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:02.429756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:02.429758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:02.430255 (+ 497us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:02.430283 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:02.430300 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:02.430402 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:02.430518 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 18:59:02.430583 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 18:59:02.435693 1325 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:02.436875 1325 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:02.443002 1325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2744ff5a65024c838924b969dbdee6d8: Generated new TSK 0
I20260812 18:59:02.443704 1325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 18:59:02.449755 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 1217
2026-08-12T18:59:02Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1459 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:59:02 dist-test-slave-6nmv krb5kdc[1340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:59:02 dist-test-slave-6nmv krb5kdc[1340](info): set up 2 sockets
Aug 12 18:59:02 dist-test-slave-6nmv krb5kdc[1340](info): commencing operation
krb5kdc: starting...
W20260812 18:59:03.624593 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:03 dist-test-slave-6nmv krb5kdc[1340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:59:03.744119 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:46795
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:46795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:03.846895 1354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:03.847219 1354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:03.847302 1354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:03.851262 1354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:03.851347 1354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:03.851389 1354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:03.851454 1354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:03.851500 1354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:03.856670 1354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:46795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:46795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:03.857657 1354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:03.858664 1354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:03.867002 1361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:03.866986 1362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:03.866986 1364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:03.867758 1354 server_base.cc:1034] running on GCE node
I20260812 18:59:03.868278 1354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:03.869326 1354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:03.870513 1354 hybrid_clock.cc:648] HybridClock initialized: now 1786561143870485 us; error 44 us; skew 500 ppm
Aug 12 18:59:03 dist-test-slave-6nmv krb5kdc[1340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561143, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:03.873418 1354 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:59:03.874343 1354 webserver.cc:460] Webserver started at http://127.29.112.254:42727/ using document root <none> and password file <none>
I20260812 18:59:03.874902 1354 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:03.875003 1354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:03.875231 1354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:03.877148 1354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "a608f1292403438bb97e139de5791582"
format_stamp: "Formatted at 2026-08-12 18:59:03 on dist-test-slave-6nmv"
server_key: "aeef3315909ac571deada295ec002d21"
server_key_iv: "375cc342870af0cb757a0f150adc25c3"
server_key_version: "encryptionkey@0"
I20260812 18:59:03.877705 1354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "a608f1292403438bb97e139de5791582"
format_stamp: "Formatted at 2026-08-12 18:59:03 on dist-test-slave-6nmv"
server_key: "aeef3315909ac571deada295ec002d21"
server_key_iv: "375cc342870af0cb757a0f150adc25c3"
server_key_version: "encryptionkey@0"
I20260812 18:59:03.881131 1354 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 18:59:03.883265 1372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:03.884318 1354 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:03.884476 1354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "a608f1292403438bb97e139de5791582"
format_stamp: "Formatted at 2026-08-12 18:59:03 on dist-test-slave-6nmv"
server_key: "aeef3315909ac571deada295ec002d21"
server_key_iv: "375cc342870af0cb757a0f150adc25c3"
server_key_version: "encryptionkey@0"
I20260812 18:59:03.884577 1354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:03.931171 1354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:03.934160 1354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:03.934386 1354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:03.941471 1354 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:46795
I20260812 18:59:03.941747 1433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:46795 every 8 connection(s)
I20260812 18:59:03.942435 1354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:03.945103 1434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:03.949800 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 1354
I20260812 18:59:03.949952 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:03.950246 30147 external_mini_cluster.cc:1428] Setting key 84c5193fbab0ef5bf48788bfc62a070b
I20260812 18:59:03.950614 1434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582: Bootstrap starting.
I20260812 18:59:03.953363 1434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:03.954315 1434 log.cc:824] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:03.956385 1434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582: No bootstrap required, opened a new log
Aug 12 18:59:03 dist-test-slave-6nmv krb5kdc[1340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:03.960958 1434 raft_consensus.cc:348] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a608f1292403438bb97e139de5791582" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46795 } }
I20260812 18:59:03.961210 1434 raft_consensus.cc:374] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:03.961278 1434 raft_consensus.cc:729] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a608f1292403438bb97e139de5791582, State: Initialized, Role: FOLLOWER
I20260812 18:59:03.961748 1434 consensus_queue.cc:260] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a608f1292403438bb97e139de5791582" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46795 } }
I20260812 18:59:03.961939 1434 raft_consensus.cc:388] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:03.962007 1434 raft_consensus.cc:482] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:03.962092 1434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:03.963161 1434 raft_consensus.cc:504] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a608f1292403438bb97e139de5791582" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46795 } }
I20260812 18:59:03.963492 1434 leader_election.cc:302] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a608f1292403438bb97e139de5791582; no voters:
I20260812 18:59:03.963833 1434 leader_election.cc:288] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:03.963894 1439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:03.964107 1439 raft_consensus.cc:686] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [term 1 LEADER]: Becoming Leader. State: Replica: a608f1292403438bb97e139de5791582, State: Running, Role: LEADER
I20260812 18:59:03.964439 1439 consensus_queue.cc:237] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a608f1292403438bb97e139de5791582" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46795 } }
I20260812 18:59:03.965081 1434 sys_catalog.cc:564] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:03.966534 1441 sys_catalog.cc:455] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a608f1292403438bb97e139de5791582" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a608f1292403438bb97e139de5791582" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46795 } } }
I20260812 18:59:03.966552 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:03.951546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.29.112.254:46795)
0812 18:59:03.951975 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:03.951989 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:03.952023 (+ 34us) server_negotiation.cc:407] Connection header received
0812 18:59:03.952907 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:03.952930 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:03.953376 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:03.953673 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:03.954626 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:03.955597 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:03.956738 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:03.957176 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:03.957285 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:03.960034 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:03.960042 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:03.960057 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:03.960087 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:03.962367 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:03.962841 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:03.962848 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:03.962853 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:03.963017 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:03.963315 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:03.963319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:03.963322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:03.963777 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:03.964658 (+ 881us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:03.964823 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 18:59:03.965070 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":151,"threads_started":1}
I20260812 18:59:03.966740 1441 sys_catalog.cc:458] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:03.967607 1452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:03.966631 1440 sys_catalog.cc:455] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a608f1292403438bb97e139de5791582. Latest consensus state: current_term: 1 leader_uuid: "a608f1292403438bb97e139de5791582" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a608f1292403438bb97e139de5791582" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46795 } } }
I20260812 18:59:03.968258 1440 sys_catalog.cc:458] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:03.970718 1452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:03.976567 1452 catalog_manager.cc:1347] Generated new cluster ID: 49e7471bc81746b5a939192a448ddd8e
I20260812 18:59:03.976631 1452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:03.986972 1452 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:03.988258 1452 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:03.996637 1452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a608f1292403438bb97e139de5791582: Generated new TSK 0
I20260812 18:59:03.997337 1452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 18:59:04.036818 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46795
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:04.152325 1468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:04.152657 1468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:04.152701 1468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:04.156785 1468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:04.156837 1468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:04.156930 1468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:04.161963 1468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:04.162989 1468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:04.163976 1468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:04.173451 1477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:04.173540 1475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:04.173480 1474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:04.173684 1468 server_base.cc:1034] running on GCE node
I20260812 18:59:04.174212 1468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:04.174811 1468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:04.176008 1468 hybrid_clock.cc:648] HybridClock initialized: now 1786561144175997 us; error 35 us; skew 500 ppm
Aug 12 18:59:04 dist-test-slave-6nmv krb5kdc[1340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:04.178696 1468 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 18:59:04.179431 1468 webserver.cc:460] Webserver started at http://127.29.112.193:37331/ using document root <none> and password file <none>
I20260812 18:59:04.179903 1468 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:04.179958 1468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:04.180120 1468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:04.182003 1468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "cba5c228f4e64c58a09879e876685c71"
format_stamp: "Formatted at 2026-08-12 18:59:04 on dist-test-slave-6nmv"
server_key: "eb0f6d7d2eaf37c2da066b6af2c65de6"
server_key_iv: "ee7b361f5805a7abd8a608bb9b714524"
server_key_version: "encryptionkey@0"
I20260812 18:59:04.182525 1468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "cba5c228f4e64c58a09879e876685c71"
format_stamp: "Formatted at 2026-08-12 18:59:04 on dist-test-slave-6nmv"
server_key: "eb0f6d7d2eaf37c2da066b6af2c65de6"
server_key_iv: "ee7b361f5805a7abd8a608bb9b714524"
server_key_version: "encryptionkey@0"
I20260812 18:59:04.185998 1468 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 18:59:04.188443 1489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:04.189527 1468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:04.189646 1468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "cba5c228f4e64c58a09879e876685c71"
format_stamp: "Formatted at 2026-08-12 18:59:04 on dist-test-slave-6nmv"
server_key: "eb0f6d7d2eaf37c2da066b6af2c65de6"
server_key_iv: "ee7b361f5805a7abd8a608bb9b714524"
server_key_version: "encryptionkey@0"
I20260812 18:59:04.189734 1468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:04.200471 1468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:04.202873 1468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:04.203053 1468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:04.203539 1468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:04.204244 1468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:04.204285 1468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:04.204341 1468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:04.204370 1468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:04.212391 1468 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:35141
I20260812 18:59:04.212415 1620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:35141 every 8 connection(s)
I20260812 18:59:04.213231 1468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:04.213508 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 1468
I20260812 18:59:04.213661 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:04.214008 30147 external_mini_cluster.cc:1428] Setting key c125475704851de8f02c4140d8ec77cc
Aug 12 18:59:04 dist-test-slave-6nmv krb5kdc[1340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 18:59:04.228647 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:04.214985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:60745 (local address 127.29.112.254:46795)
0812 18:59:04.215120 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:04.215125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:04.216093 (+ 968us) server_negotiation.cc:407] Connection header received
0812 18:59:04.218071 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:04.218074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:04.218129 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:04.218207 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:04.219552 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.220092 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.221880 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.222139 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.222234 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:04.224955 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:04.224958 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:04.224960 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:04.224989 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:04.226822 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:04.227299 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:04.227302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:04.227303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:04.227388 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:04.227689 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:04.227691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:04.227692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:04.227861 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:04.227944 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:04.228390 (+ 446us) server_negotiation.cc:299] Negotiation successful
0812 18:59:04.228496 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
I20260812 18:59:04.229410 1623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:04.215401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:46795 (local address 127.29.112.193:60745)
0812 18:59:04.215877 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:04.215964 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:04.217870 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:04.218364 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:04.218371 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:04.218716 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:04.219388 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:04.219420 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.220227 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.220233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:04.221685 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:04.221700 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.222326 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.222332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:04.222537 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:04.223096 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:04.223115 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:04.224760 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:04.226932 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:04.226937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:04.226948 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:04.227188 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:04.227485 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:04.227487 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:04.227488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:04.227608 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:04.227957 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:04.227962 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:04.228156 (+ 194us) client_negotiation.cc:769] Sending connection context
0812 18:59:04.228298 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 18:59:04.228568 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":113,"threads_started":1}
I20260812 18:59:04.230629 1621 heartbeater.cc:347] Connected to a master server at 127.29.112.254:46795
I20260812 18:59:04.230870 1621 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:04.231314 1621 heartbeater.cc:511] Master 127.29.112.254:46795 requested a full tablet report, sending...
I20260812 18:59:04.232757 1391 ts_manager.cc:194] Registered new tserver with Master: cba5c228f4e64c58a09879e876685c71 (127.29.112.193:35141)
I20260812 18:59:04.233908 1391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:60745
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 18:59:04.244928 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46795
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:04.348853 1627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:04.349071 1627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:04.349107 1627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:04.352962 1627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:04.353006 1627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:04.353081 1627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:04.357558 1627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:04.358390 1627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:04.359207 1627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:04.367614 1636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:04.367628 1637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:04.367714 1640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:04.367777 1627 server_base.cc:1034] running on GCE node
I20260812 18:59:04.368178 1627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:04.368806 1627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:04.369963 1627 hybrid_clock.cc:648] HybridClock initialized: now 1786561144369952 us; error 22 us; skew 500 ppm
Aug 12 18:59:04 dist-test-slave-6nmv krb5kdc[1340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:04.372615 1627 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 18:59:04.373425 1627 webserver.cc:460] Webserver started at http://127.29.112.194:33837/ using document root <none> and password file <none>
I20260812 18:59:04.373950 1627 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:04.374009 1627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:04.374186 1627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:04.376049 1627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "2694731d72ba4b0bbaeaa0cb04ece0bd"
format_stamp: "Formatted at 2026-08-12 18:59:04 on dist-test-slave-6nmv"
server_key: "d749d03c60d6a790f94adb7194afdb58"
server_key_iv: "d6267b25f74ddb5b71bdc60bcf6c8762"
server_key_version: "encryptionkey@0"
I20260812 18:59:04.376547 1627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "2694731d72ba4b0bbaeaa0cb04ece0bd"
format_stamp: "Formatted at 2026-08-12 18:59:04 on dist-test-slave-6nmv"
server_key: "d749d03c60d6a790f94adb7194afdb58"
server_key_iv: "d6267b25f74ddb5b71bdc60bcf6c8762"
server_key_version: "encryptionkey@0"
I20260812 18:59:04.379828 1627 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 18:59:04.381989 1648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:04.383026 1627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:04.383144 1627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "2694731d72ba4b0bbaeaa0cb04ece0bd"
format_stamp: "Formatted at 2026-08-12 18:59:04 on dist-test-slave-6nmv"
server_key: "d749d03c60d6a790f94adb7194afdb58"
server_key_iv: "d6267b25f74ddb5b71bdc60bcf6c8762"
server_key_version: "encryptionkey@0"
I20260812 18:59:04.383250 1627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:04.398285 1627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:04.400861 1627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:04.401036 1627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:04.401527 1627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:04.402249 1627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:04.402328 1627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:04.402402 1627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:04.402443 1627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:04.410838 1627 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:44349
I20260812 18:59:04.410883 1780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:44349 every 8 connection(s)
I20260812 18:59:04.411589 1627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
Aug 12 18:59:04 dist-test-slave-6nmv krb5kdc[1340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:04.420691 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 1627
I20260812 18:59:04.420822 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:04.421099 30147 external_mini_cluster.cc:1428] Setting key fd63fa164afc8dbad360f15bbe85f172
I20260812 18:59:04.424836 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:04.413253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:46111 (local address 127.29.112.254:46795)
0812 18:59:04.413394 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:04.413399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:04.414239 (+ 840us) server_negotiation.cc:407] Connection header received
0812 18:59:04.415524 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:04.415527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:04.415574 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:04.415649 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:04.416750 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.417283 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.418425 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.418630 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.418709 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:04.421077 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:04.421080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:04.421083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:04.421105 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:04.422802 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:04.423305 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:04.423311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:04.423313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:04.423407 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:04.423746 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:04.423751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:04.423753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:04.423937 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:04.424057 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:04.424565 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 18:59:04.424700 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 18:59:04.425884 1783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:04.413595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:46795 (local address 127.29.112.194:46111)
0812 18:59:04.414092 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:04.414132 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:04.415334 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:04.415804 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:04.415810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:04.416167 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:04.416570 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:04.416606 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.417385 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.417387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:04.418315 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:04.418324 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.418749 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.418751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:04.418868 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:04.419461 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:04.419476 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:04.420876 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:04.422924 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:04.422929 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:04.422942 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:04.423197 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:04.423507 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:04.423510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:04.423512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:04.423648 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:04.424056 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:04.424062 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:04.424275 (+ 213us) client_negotiation.cc:769] Sending connection context
0812 18:59:04.424473 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 18:59:04.424844 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":111,"threads_started":1}
I20260812 18:59:04.427008 1781 heartbeater.cc:347] Connected to a master server at 127.29.112.254:46795
I20260812 18:59:04.427239 1781 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:04.427744 1781 heartbeater.cc:511] Master 127.29.112.254:46795 requested a full tablet report, sending...
I20260812 18:59:04.428925 1391 ts_manager.cc:194] Registered new tserver with Master: 2694731d72ba4b0bbaeaa0cb04ece0bd (127.29.112.194:44349)
I20260812 18:59:04.429508 1391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:46111
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 18:59:04.450771 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46795
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:04.557487 1787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:04.557713 1787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:04.557754 1787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:04.561260 1787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:04.561307 1787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:04.561388 1787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:04.566218 1787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:04.567044 1787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:04.567860 1787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:04.576903 1795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:04.576924 1797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:04.576924 1794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:04.577157 1787 server_base.cc:1034] running on GCE node
I20260812 18:59:04.577613 1787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:04.578282 1787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:04.579483 1787 hybrid_clock.cc:648] HybridClock initialized: now 1786561144579476 us; error 29 us; skew 500 ppm
Aug 12 18:59:04 dist-test-slave-6nmv krb5kdc[1340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:04.582224 1787 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 18:59:04.583042 1787 webserver.cc:460] Webserver started at http://127.29.112.195:42603/ using document root <none> and password file <none>
I20260812 18:59:04.583508 1787 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:04.583563 1787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:04.583731 1787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:04.585557 1787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "01d470dd93cf4616bd21be0933fc1496"
format_stamp: "Formatted at 2026-08-12 18:59:04 on dist-test-slave-6nmv"
server_key: "50706c1e361539cfc1b74fb22310f5bd"
server_key_iv: "27142a1a7144ec829254e5b678c3a4d8"
server_key_version: "encryptionkey@0"
I20260812 18:59:04.586144 1787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "01d470dd93cf4616bd21be0933fc1496"
format_stamp: "Formatted at 2026-08-12 18:59:04 on dist-test-slave-6nmv"
server_key: "50706c1e361539cfc1b74fb22310f5bd"
server_key_iv: "27142a1a7144ec829254e5b678c3a4d8"
server_key_version: "encryptionkey@0"
I20260812 18:59:04.589800 1787 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 18:59:04.592104 1804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:04.593295 1787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 18:59:04.593417 1787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "01d470dd93cf4616bd21be0933fc1496"
format_stamp: "Formatted at 2026-08-12 18:59:04 on dist-test-slave-6nmv"
server_key: "50706c1e361539cfc1b74fb22310f5bd"
server_key_iv: "27142a1a7144ec829254e5b678c3a4d8"
server_key_version: "encryptionkey@0"
I20260812 18:59:04.593518 1787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:04.610074 1787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:04.613204 1787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:04.613346 1787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:04.613888 1787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:04.614691 1787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:04.614737 1787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:04.614800 1787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:04.614864 1787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:04.622869 1787 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:39401
I20260812 18:59:04.622937 1937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:39401 every 8 connection(s)
I20260812 18:59:04.623680 1787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:59:04.626602 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 1787
I20260812 18:59:04.626724 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:04.627010 30147 external_mini_cluster.cc:1428] Setting key 7a5a46341c3f13e5eb9d6598093adf97
Aug 12 18:59:04 dist-test-slave-6nmv krb5kdc[1340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:04.636876 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:04.625347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:49291 (local address 127.29.112.254:46795)
0812 18:59:04.625496 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:04.625501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:04.626745 (+ 1244us) server_negotiation.cc:407] Connection header received
0812 18:59:04.628266 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:04.628272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:04.628335 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:04.628420 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:04.629609 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.630252 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.631341 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.631565 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.631664 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:04.634008 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:04.634011 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:04.634013 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:04.634037 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:04.635272 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:04.635726 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:04.635729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:04.635730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:04.635806 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:04.636108 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:04.636110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:04.636111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:04.636250 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:04.636314 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:04.636694 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 18:59:04.636777 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 18:59:04.637624 1941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:04.625722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:46795 (local address 127.29.112.195:49291)
0812 18:59:04.626566 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:04.626623 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:04.628032 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:04.628597 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:04.628603 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:04.628925 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:04.629476 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:04.629490 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.630388 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.630391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:04.631225 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:04.631234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.631694 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.631696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:04.631826 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:04.632369 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:04.632384 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:04.633850 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:04.635375 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:04.635379 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:04.635390 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:04.635640 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:04.635883 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:04.635885 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:04.635887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:04.636024 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:04.636326 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:04.636330 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:04.636522 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 18:59:04.636654 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 18:59:04.636854 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"spinlock_wait_cycles":353664,"thread_start_us":111,"threads_started":1}
I20260812 18:59:04.638664 1938 heartbeater.cc:347] Connected to a master server at 127.29.112.254:46795
I20260812 18:59:04.638883 1938 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:04.639258 1938 heartbeater.cc:511] Master 127.29.112.254:46795 requested a full tablet report, sending...
I20260812 18:59:04.640282 1391 ts_manager.cc:194] Registered new tserver with Master: 01d470dd93cf4616bd21be0933fc1496 (127.29.112.195:39401)
I20260812 18:59:04.640816 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 18:59:04.640947 1391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:49291
Aug 12 18:59:04 dist-test-slave-6nmv krb5kdc[1340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561144, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 18:59:04 dist-test-slave-6nmv krb5kdc[1340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561144, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:04.666383 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:04.657558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38770 (local address 127.29.112.254:46795)
0812 18:59:04.657725 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:04.657731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:04.657957 (+ 226us) server_negotiation.cc:407] Connection header received
0812 18:59:04.658124 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:04.658142 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:04.658189 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:04.658302 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:04.659025 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.659542 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.660461 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:04.660685 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:04.660807 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:04.663273 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:04.663276 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:04.663278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:04.663298 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:04.664503 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:04.664959 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:04.664964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:04.664966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:04.665051 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:04.665360 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:04.665363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:04.665365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:04.665581 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:04.665726 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:04.666048 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 18:59:04.666185 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 18:59:04.666698 1391 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38770
I20260812 18:59:04.667646 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 1468
I20260812 18:59:04.682480 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 1627
I20260812 18:59:04.692711 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 1787
I20260812 18:59:04.701396 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 1354
2026-08-12T18:59:04Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2250 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:59:04 dist-test-slave-6nmv krb5kdc[1953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:59:04 dist-test-slave-6nmv krb5kdc[1953](info): set up 2 sockets
Aug 12 18:59:04 dist-test-slave-6nmv krb5kdc[1953](info): commencing operation
krb5kdc: starting...
W20260812 18:59:05.786240 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:05 dist-test-slave-6nmv krb5kdc[1953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:59:05.906809 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:37173
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:35119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:37173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:06.011927 1970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:06.012174 1970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:06.012213 1970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:06.016100 1970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:06.016162 1970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:06.016180 1970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:06.016196 1970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:06.016251 1970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:06.021023 1970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:35119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:37173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:37173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:06.022040 1970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:06.022998 1970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:06.031179 1977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:06.031301 1970 server_base.cc:1034] running on GCE node
W20260812 18:59:06.031220 1978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:06.031193 1980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:06.031986 1970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:06.032980 1970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:06.034183 1970 hybrid_clock.cc:648] HybridClock initialized: now 1786561146034164 us; error 27 us; skew 500 ppm
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[1953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:06.036835 1970 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:59:06.037683 1970 webserver.cc:460] Webserver started at http://127.29.112.254:38713/ using document root <none> and password file <none>
I20260812 18:59:06.038264 1970 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:06.038357 1970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:06.038586 1970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:06.040484 1970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "7129676bc94d436bb3ac67918ab0c335"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "76d2c09cbc44f5f16b33538013af52b9"
server_key_iv: "ddebcc9f54d0de62e743c181b94050b9"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.041034 1970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "7129676bc94d436bb3ac67918ab0c335"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "76d2c09cbc44f5f16b33538013af52b9"
server_key_iv: "ddebcc9f54d0de62e743c181b94050b9"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.044294 1970 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 18:59:06.046576 1991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:06.047612 1970 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:06.047741 1970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "7129676bc94d436bb3ac67918ab0c335"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "76d2c09cbc44f5f16b33538013af52b9"
server_key_iv: "ddebcc9f54d0de62e743c181b94050b9"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.047864 1970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:06.063993 1970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:06.066732 1970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:06.066936 1970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:06.073983 1970 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:37173
I20260812 18:59:06.074011 2055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:37173 every 8 connection(s)
I20260812 18:59:06.074862 1970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:06.077421 2056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:06.082754 2056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335: Bootstrap starting.
I20260812 18:59:06.082916 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 1970
I20260812 18:59:06.083072 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:06.083343 30147 external_mini_cluster.cc:1428] Setting key 5cf8eab6966edfdb411979aa39857893
I20260812 18:59:06.085472 2056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:06.086383 2056 log.cc:824] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:06.088390 2056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335: No bootstrap required, opened a new log
I20260812 18:59:06.092236 2056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7129676bc94d436bb3ac67918ab0c335" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37173 } }
I20260812 18:59:06.092499 2056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:06.092602 2056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7129676bc94d436bb3ac67918ab0c335, State: Initialized, Role: FOLLOWER
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[1953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:06.093104 2056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7129676bc94d436bb3ac67918ab0c335" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37173 } }
I20260812 18:59:06.093268 2056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:06.093350 2056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:06.093472 2056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:06.094393 2056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7129676bc94d436bb3ac67918ab0c335" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37173 } }
I20260812 18:59:06.094714 2056 leader_election.cc:302] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7129676bc94d436bb3ac67918ab0c335; no voters:
I20260812 18:59:06.095063 2056 leader_election.cc:288] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:06.095130 2062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:06.095330 2062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [term 1 LEADER]: Becoming Leader. State: Replica: 7129676bc94d436bb3ac67918ab0c335, State: Running, Role: LEADER
I20260812 18:59:06.095674 2062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7129676bc94d436bb3ac67918ab0c335" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37173 } }
I20260812 18:59:06.096274 2056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:06.097275 2063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7129676bc94d436bb3ac67918ab0c335. Latest consensus state: current_term: 1 leader_uuid: "7129676bc94d436bb3ac67918ab0c335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7129676bc94d436bb3ac67918ab0c335" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37173 } } }
I20260812 18:59:06.097774 2063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:06.097651 2068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7129676bc94d436bb3ac67918ab0c335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7129676bc94d436bb3ac67918ab0c335" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37173 } } }
I20260812 18:59:06.098100 2068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:06.098995 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:06.084647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.29.112.254:37173)
0812 18:59:06.085070 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:06.085096 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:06.085137 (+ 41us) server_negotiation.cc:407] Connection header received
0812 18:59:06.086022 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:06.086044 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:06.086465 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:06.086795 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:06.087563 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.088928 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.089722 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.090423 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.090585 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:06.093437 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:06.093445 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:06.093460 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:06.093496 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:06.095459 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.095988 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.095993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.095998 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.096143 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.096457 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.096462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.096468 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.096953 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:06.097161 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:06.097887 (+ 726us) server_negotiation.cc:299] Negotiation successful
0812 18:59:06.098185 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":117,"threads_started":1}
W20260812 18:59:06.101053 2084 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 18:59:06.101173 2084 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 18:59:06.101274 2079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:06.101982 2079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:06.107097 2079 catalog_manager.cc:1347] Generated new cluster ID: 644fd99fff404ce3ab335ea194971f71
I20260812 18:59:06.107189 2079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:06.123123 2079 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:06.123878 2079 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:06.130487 2079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7129676bc94d436bb3ac67918ab0c335: Generated new TSK 0
I20260812 18:59:06.131060 2079 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 18:59:06.160706 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37173
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:35119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:06.267225 2092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:06.267465 2092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:06.267504 2092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:06.271265 2092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:06.271311 2092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:06.271363 2092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:06.276446 2092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:35119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:06.277256 2092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:06.278138 2092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:06.287173 2100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:06.287204 2103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:06.287241 2092 server_base.cc:1034] running on GCE node
W20260812 18:59:06.287164 2106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:06.287814 2092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:06.288380 2092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:06.289554 2092 hybrid_clock.cc:648] HybridClock initialized: now 1786561146289539 us; error 31 us; skew 500 ppm
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[1953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:06.292280 2092 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 18:59:06.293125 2092 webserver.cc:460] Webserver started at http://127.29.112.193:33741/ using document root <none> and password file <none>
I20260812 18:59:06.293687 2092 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:06.293776 2092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:06.294023 2092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:06.295990 2092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "6fb68fee5e664cf08863e7cecd59e665"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "873881d6b52c16edbddc0062e3ebdaff"
server_key_iv: "8c92c7a7db6e761b0ae9b7586918c4a6"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.296605 2092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "6fb68fee5e664cf08863e7cecd59e665"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "873881d6b52c16edbddc0062e3ebdaff"
server_key_iv: "8c92c7a7db6e761b0ae9b7586918c4a6"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.300063 2092 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 18:59:06.302294 2115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:06.303452 2092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:06.303584 2092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "6fb68fee5e664cf08863e7cecd59e665"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "873881d6b52c16edbddc0062e3ebdaff"
server_key_iv: "8c92c7a7db6e761b0ae9b7586918c4a6"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.303686 2092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:06.317590 2092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:06.320701 2092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:06.320895 2092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:06.321461 2092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:06.322326 2092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:06.322407 2092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:06.322496 2092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:06.322551 2092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:06.331575 2092 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:34619
I20260812 18:59:06.331691 2252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:34619 every 8 connection(s)
I20260812 18:59:06.332484 2092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:06.336688 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 2092
I20260812 18:59:06.336828 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:06.337165 30147 external_mini_cluster.cc:1428] Setting key ad12abfc9f063cc797f62a48c9c1f0d5
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[1953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:06.346923 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:06.334400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:55821 (local address 127.29.112.254:37173)
0812 18:59:06.334548 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:06.334555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:06.335378 (+ 823us) server_negotiation.cc:407] Connection header received
0812 18:59:06.336694 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:06.336700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:06.336766 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:06.336895 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:06.338260 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.339033 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.340112 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.340516 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.340647 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:06.343337 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:06.343340 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:06.343342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:06.343364 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:06.344854 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.345405 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.345410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.345412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.345493 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.345851 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.345855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.345856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.346049 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:06.346169 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:06.346646 (+ 477us) server_negotiation.cc:299] Negotiation successful
0812 18:59:06.346761 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 18:59:06.347739 2255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:06.334737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:37173 (local address 127.29.112.193:55821)
0812 18:59:06.335209 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:06.335249 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:06.336476 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:06.337057 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:06.337064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:06.337475 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:06.338105 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:06.338124 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.339158 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.339161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:06.339968 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:06.339978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.340718 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.340726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:06.340951 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:06.341605 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:06.341625 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:06.343180 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:06.345022 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:06.345027 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:06.345041 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:06.345299 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:06.345588 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:06.345592 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:06.345594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:06.345735 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:06.346181 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:06.346188 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:06.346425 (+ 237us) client_negotiation.cc:769] Sending connection context
0812 18:59:06.346606 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 18:59:06.346893 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":131,"threads_started":1}
I20260812 18:59:06.349124 2253 heartbeater.cc:347] Connected to a master server at 127.29.112.254:37173
I20260812 18:59:06.349349 2253 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:06.349905 2253 heartbeater.cc:511] Master 127.29.112.254:37173 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
I20260812 18:59:06.351401 2009 ts_manager.cc:194] Registered new tserver with Master: 6fb68fee5e664cf08863e7cecd59e665 (127.29.112.193:34619)
I20260812 18:59:06.352587 2009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:55821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 18:59:06.365137 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37173
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:35119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:06.470921 2258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:06.471148 2258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:06.471218 2258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:06.475168 2258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:06.475217 2258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:06.475301 2258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:06.480041 2258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:35119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:06.480995 2258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:06.481961 2258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:06.490747 2264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:06.490756 2267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:06.490756 2263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:06.491120 2258 server_base.cc:1034] running on GCE node
I20260812 18:59:06.491698 2258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:06.492296 2258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:06.493441 2258 hybrid_clock.cc:648] HybridClock initialized: now 1786561146493364 us; error 92 us; skew 500 ppm
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[1953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:06.496223 2258 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 18:59:06.497128 2258 webserver.cc:460] Webserver started at http://127.29.112.194:34775/ using document root <none> and password file <none>
I20260812 18:59:06.497763 2258 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:06.497877 2258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:06.498095 2258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:06.500465 2258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "d09f5b1b345c47898dfc0247603150fb"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "ad0cd3ebbb7e5d0b6a45423df6f6102d"
server_key_iv: "ed268c7ce2d145599c92b4cdb6b4f9f9"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.501046 2258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "d09f5b1b345c47898dfc0247603150fb"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "ad0cd3ebbb7e5d0b6a45423df6f6102d"
server_key_iv: "ed268c7ce2d145599c92b4cdb6b4f9f9"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.504511 2258 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260812 18:59:06.506884 2275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:06.508112 2258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:06.508237 2258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "d09f5b1b345c47898dfc0247603150fb"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "ad0cd3ebbb7e5d0b6a45423df6f6102d"
server_key_iv: "ed268c7ce2d145599c92b4cdb6b4f9f9"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.508337 2258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:06.519634 2258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:06.522630 2258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:06.522817 2258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:06.523409 2258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:06.524220 2258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:06.524284 2258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:06.524348 2258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:06.524394 2258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:06.532944 2258 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:34467
I20260812 18:59:06.532974 2404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:34467 every 8 connection(s)
I20260812 18:59:06.533735 2258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:06.541258 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 2258
I20260812 18:59:06.541386 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:06.541729 30147 external_mini_cluster.cc:1428] Setting key 8726f9c191547721406f6817dcdc3a07
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[1953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:06.547067 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:06.535439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:58901 (local address 127.29.112.254:37173)
0812 18:59:06.535595 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:06.535599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:06.536408 (+ 809us) server_negotiation.cc:407] Connection header received
0812 18:59:06.537719 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:06.537722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:06.537767 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:06.537901 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:06.538948 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.539404 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.540381 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.540575 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.540644 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:06.543296 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:06.543299 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:06.543303 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:06.543326 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:06.544896 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.545413 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.545418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.545422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.545536 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.545981 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.545983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.545985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.546248 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:06.546325 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:06.546872 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 18:59:06.546958 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 18:59:06.548131 2407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:06.535802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:37173 (local address 127.29.112.194:58901)
0812 18:59:06.536250 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:06.536287 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:06.537510 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:06.538035 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:06.538041 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:06.538367 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:06.538826 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:06.538838 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.539497 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.539499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:06.540283 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:06.540292 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.540687 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.540689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:06.540810 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:06.541300 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:06.541320 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:06.543141 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:06.545011 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:06.545015 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:06.545046 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:06.545285 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:06.545645 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:06.545649 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:06.545654 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:06.545886 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:06.546351 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:06.546359 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:06.546646 (+ 287us) client_negotiation.cc:769] Sending connection context
0812 18:59:06.546837 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 18:59:06.547147 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":118,"threads_started":1}
I20260812 18:59:06.549223 2405 heartbeater.cc:347] Connected to a master server at 127.29.112.254:37173
I20260812 18:59:06.549446 2405 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:06.549924 2405 heartbeater.cc:511] Master 127.29.112.254:37173 requested a full tablet report, sending...
I20260812 18:59:06.550926 2009 ts_manager.cc:194] Registered new tserver with Master: d09f5b1b345c47898dfc0247603150fb (127.29.112.194:34467)
I20260812 18:59:06.551486 2009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:58901
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 18:59:06.569248 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37173
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:35119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:06.679850 2410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:06.680116 2410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:06.680194 2410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:06.684139 2410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:06.684209 2410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:06.684290 2410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:06.689486 2410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:35119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:06.690439 2410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:06.691306 2410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:06.700407 2418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:06.700420 2415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:06.700469 2410 server_base.cc:1034] running on GCE node
W20260812 18:59:06.700435 2416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:06.700958 2410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:06.701493 2410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:06.702663 2410 hybrid_clock.cc:648] HybridClock initialized: now 1786561146702641 us; error 41 us; skew 500 ppm
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[1953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:06.705096 2410 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 18:59:06.705964 2410 webserver.cc:460] Webserver started at http://127.29.112.195:37939/ using document root <none> and password file <none>
I20260812 18:59:06.706575 2410 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:06.706662 2410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:06.706878 2410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:06.708781 2410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "13006c979ed2455fbbf54fcc3ef3c7e2"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "a5c071d9f7aec1358f4ad99899cd0727"
server_key_iv: "5a76c1a7cd2f1fadbbad14f67aec495a"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.709268 2410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "13006c979ed2455fbbf54fcc3ef3c7e2"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "a5c071d9f7aec1358f4ad99899cd0727"
server_key_iv: "5a76c1a7cd2f1fadbbad14f67aec495a"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.712497 2410 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 18:59:06.714740 2425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:06.715869 2410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:06.715996 2410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "13006c979ed2455fbbf54fcc3ef3c7e2"
format_stamp: "Formatted at 2026-08-12 18:59:06 on dist-test-slave-6nmv"
server_key: "a5c071d9f7aec1358f4ad99899cd0727"
server_key_iv: "5a76c1a7cd2f1fadbbad14f67aec495a"
server_key_version: "encryptionkey@0"
I20260812 18:59:06.716107 2410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:06.723723 2410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:06.726240 2410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:06.726428 2410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:06.726946 2410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:06.727659 2410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:06.727730 2410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:06.727792 2410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:06.727842 2410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:06.736673 2410 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:36369
I20260812 18:59:06.736690 2550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:36369 every 8 connection(s)
I20260812 18:59:06.737450 2410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[1953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:06.746877 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 2410
I20260812 18:59:06.747005 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:06.747280 30147 external_mini_cluster.cc:1428] Setting key 8fea5bf3dd84eb1fa560f3b2b3e72d0d
I20260812 18:59:06.750404 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:06.739163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:42503 (local address 127.29.112.254:37173)
0812 18:59:06.739323 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:06.739326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:06.740039 (+ 713us) server_negotiation.cc:407] Connection header received
0812 18:59:06.741304 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:06.741307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:06.741361 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:06.741473 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:06.742533 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.743046 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.744108 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.744315 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.744427 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:06.746763 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:06.746766 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:06.746768 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:06.746795 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:06.748552 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.749027 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.749030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.749032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.749121 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.749502 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.749504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.749505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.749667 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:06.749740 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:06.750194 (+ 454us) server_negotiation.cc:299] Negotiation successful
0812 18:59:06.750294 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 18:59:06.751174 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:06.739497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:37173 (local address 127.29.112.195:42503)
0812 18:59:06.739892 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:06.739947 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:06.741112 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:06.741621 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:06.741627 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:06.741986 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:06.742377 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:06.742407 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.743170 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.743178 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:06.743962 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:06.743971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.744461 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.744464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:06.744626 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:06.745216 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:06.745231 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:06.746612 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:06.748662 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:06.748669 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:06.748683 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:06.748936 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:06.749213 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:06.749219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:06.749223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:06.749406 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:06.749750 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:06.749754 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:06.750001 (+ 247us) client_negotiation.cc:769] Sending connection context
0812 18:59:06.750146 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 18:59:06.750378 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":95,"threads_started":1}
I20260812 18:59:06.752161 2551 heartbeater.cc:347] Connected to a master server at 127.29.112.254:37173
I20260812 18:59:06.752368 2551 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:06.752775 2551 heartbeater.cc:511] Master 127.29.112.254:37173 requested a full tablet report, sending...
I20260812 18:59:06.753671 2009 ts_manager.cc:194] Registered new tserver with Master: 13006c979ed2455fbbf54fcc3ef3c7e2 (127.29.112.195:36369)
I20260812 18:59:06.754223 2009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:42503
I20260812 18:59:06.760792 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[1953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[1953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.193@KRBTEST.COM
I20260812 18:59:06.785207 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:06.776460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34842 (local address 127.29.112.193:34619)
0812 18:59:06.776876 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:06.776893 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:06.776914 (+ 21us) server_negotiation.cc:407] Connection header received
0812 18:59:06.776957 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:06.776961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:06.777120 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:06.777267 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:06.778035 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.778752 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.779550 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.779904 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.780016 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:06.782269 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:06.782276 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:06.782287 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:06.782309 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:06.783700 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.784078 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.784082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.784083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.784157 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.784390 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.784392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.784393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.784594 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:06.784739 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:06.784905 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 18:59:06.785037 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":157,"threads_started":1}
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[1953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:06.795735 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:06.787944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35642 (local address 127.29.112.254:37173)
0812 18:59:06.788078 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:06.788083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:06.788127 (+ 44us) server_negotiation.cc:407] Connection header received
0812 18:59:06.788283 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:06.788287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:06.788337 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:06.788425 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:06.788965 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.789475 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.790276 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:06.790490 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:06.790592 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:06.792766 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:06.792769 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:06.792771 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:06.792795 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:06.794257 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.794625 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.794627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.794628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.794702 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:06.794919 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:06.794922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:06.794924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:06.795142 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:06.795238 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:06.795430 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 18:59:06.795552 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 18:59:06.796139 2009 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35642
I20260812 18:59:06.796744 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 2092
I20260812 18:59:06.808223 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 2258
I20260812 18:59:06.822948 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 2410
I20260812 18:59:06.830097 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 1970
2026-08-12T18:59:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2128 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[2564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[2564](info): set up 2 sockets
Aug 12 18:59:06 dist-test-slave-6nmv krb5kdc[2564](info): commencing operation
krb5kdc: starting...
W20260812 18:59:07.955379 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:08 dist-test-slave-6nmv krb5kdc[2564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:59:08.074648 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:42151
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:33495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:42151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:08.178619 2578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:08.178834 2578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:08.178870 2578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:08.182574 2578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:08.182615 2578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:08.182632 2578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:08.182679 2578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:08.182709 2578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:08.187455 2578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:33495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:42151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:42151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:08.188344 2578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:08.189257 2578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:08.197016 2584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:08.197026 2583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:08.197042 2588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:08.197358 2578 server_base.cc:1034] running on GCE node
I20260812 18:59:08.197912 2578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:08.198853 2578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:08.200014 2578 hybrid_clock.cc:648] HybridClock initialized: now 1786561148199996 us; error 29 us; skew 500 ppm
Aug 12 18:59:08 dist-test-slave-6nmv krb5kdc[2564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:08.202862 2578 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:59:08.203568 2578 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:08.203662 2578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:08.203879 2578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:08.205714 2578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "5052a77dd322433685cd626a20ff4b57"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "80a6ca4270ae9efc383eb415170461db"
server_key_iv: "c44976a6ba74717eecc63e10d7cd2453"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.206341 2578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "5052a77dd322433685cd626a20ff4b57"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "80a6ca4270ae9efc383eb415170461db"
server_key_iv: "c44976a6ba74717eecc63e10d7cd2453"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.209649 2578 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 18:59:08.211772 2594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:08.212842 2578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:08.213001 2578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "5052a77dd322433685cd626a20ff4b57"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "80a6ca4270ae9efc383eb415170461db"
server_key_iv: "c44976a6ba74717eecc63e10d7cd2453"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.213104 2578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:08.227300 2578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:08.230141 2578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:08.230345 2578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:08.237287 2578 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:42151
I20260812 18:59:08.237551 2651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:42151 every 8 connection(s)
I20260812 18:59:08.237980 2578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:08.240156 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 2578
I20260812 18:59:08.240293 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:08.240554 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:08.240562 30147 external_mini_cluster.cc:1428] Setting key aa8ce0685a84b4d612149e3f3d2e4bf1
I20260812 18:59:08.247381 2652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57: Bootstrap starting.
I20260812 18:59:08.249661 2652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57: Neither blocks nor log segments found. Creating new log.
Aug 12 18:59:08 dist-test-slave-6nmv krb5kdc[2564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:08.250574 2652 log.cc:824] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:08.252579 2652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57: No bootstrap required, opened a new log
I20260812 18:59:08.255002 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.241953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33854 (local address 127.29.112.254:42151)
0812 18:59:08.242394 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:08.242411 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:08.242450 (+ 39us) server_negotiation.cc:407] Connection header received
0812 18:59:08.243228 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:08.243251 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:08.243669 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:08.243995 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:08.244684 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.246161 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.247054 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.247690 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.247870 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.250437 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:08.250442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:08.250454 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:08.250492 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:08.252316 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:08.252843 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:08.252848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:08.252852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:08.253004 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:08.253294 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:08.253296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:08.253297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:08.253624 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:08.253763 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:08.254048 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 18:59:08.254319 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":142,"threads_started":1}
I20260812 18:59:08.256383 2652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5052a77dd322433685cd626a20ff4b57" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42151 } }
I20260812 18:59:08.256644 2652 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:08.256700 2652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5052a77dd322433685cd626a20ff4b57, State: Initialized, Role: FOLLOWER
I20260812 18:59:08.257093 2652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5052a77dd322433685cd626a20ff4b57" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42151 } }
I20260812 18:59:08.257238 2652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:08.257315 2652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:08.257427 2652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:08.258235 2652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5052a77dd322433685cd626a20ff4b57" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42151 } }
I20260812 18:59:08.258549 2652 leader_election.cc:302] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5052a77dd322433685cd626a20ff4b57; no voters:
I20260812 18:59:08.258849 2652 leader_election.cc:288] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:08.258917 2657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:08.259147 2657 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [term 1 LEADER]: Becoming Leader. State: Replica: 5052a77dd322433685cd626a20ff4b57, State: Running, Role: LEADER
I20260812 18:59:08.259402 2657 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5052a77dd322433685cd626a20ff4b57" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42151 } }
I20260812 18:59:08.260030 2652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:08.261054 2660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5052a77dd322433685cd626a20ff4b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5052a77dd322433685cd626a20ff4b57" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42151 } } }
I20260812 18:59:08.261186 2660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:08.261510 2658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5052a77dd322433685cd626a20ff4b57. Latest consensus state: current_term: 1 leader_uuid: "5052a77dd322433685cd626a20ff4b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5052a77dd322433685cd626a20ff4b57" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42151 } } }
I20260812 18:59:08.261597 2658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:08.261626 2667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:08.264498 2667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:08.269711 2667 catalog_manager.cc:1347] Generated new cluster ID: 4f8e05ef4ff1434a8196fd5f906e4737
I20260812 18:59:08.269775 2667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:08.274950 2667 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:08.276206 2667 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:08.286264 2667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57: Generated new TSK 0
I20260812 18:59:08.287039 2667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 18:59:08.316705 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:42151
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:33495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:08.425521 2680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:08.425750 2680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:08.425792 2680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:08.429664 2680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:08.429718 2680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:08.429805 2680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:08.434795 2680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:33495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:08.435742 2680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:08.436697 2680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:08.445448 2685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:08.445433 2680 server_base.cc:1034] running on GCE node
W20260812 18:59:08.445384 2688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:08.445389 2686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:08.446065 2680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:08.446630 2680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:08.447799 2680 hybrid_clock.cc:648] HybridClock initialized: now 1786561148447782 us; error 31 us; skew 500 ppm
Aug 12 18:59:08 dist-test-slave-6nmv krb5kdc[2564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:08.450659 2680 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 18:59:08.451298 2680 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:08.451356 2680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:08.451500 2680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:08.453392 2680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "6f9a02345d674eeb8a617294ba4e73f1"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "d4ff3c9e194be52104ca4b2a4bb7e80c"
server_key_iv: "1c80ab890fe17f1b8ef3631ec10c9c78"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.453941 2680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "6f9a02345d674eeb8a617294ba4e73f1"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "d4ff3c9e194be52104ca4b2a4bb7e80c"
server_key_iv: "1c80ab890fe17f1b8ef3631ec10c9c78"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.457296 2680 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 18:59:08.459654 2697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:08.460930 2680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 18:59:08.461048 2680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "6f9a02345d674eeb8a617294ba4e73f1"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "d4ff3c9e194be52104ca4b2a4bb7e80c"
server_key_iv: "1c80ab890fe17f1b8ef3631ec10c9c78"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.461139 2680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:08.486590 2680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:08.489671 2680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:08.489907 2680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:08.490518 2680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:08.491299 2680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:08.491343 2680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:08.491407 2680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:08.491434 2680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:08.500602 2680 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:45799
I20260812 18:59:08.500625 2836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:45799 every 8 connection(s)
I20260812 18:59:08.501119 2680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:08.502835 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 2680
I20260812 18:59:08.502969 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:08.503247 30147 external_mini_cluster.cc:1428] Setting key fed516b43361cf0b2ee06100619dc226
Aug 12 18:59:08 dist-test-slave-6nmv krb5kdc[2564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 18:59:08.514870 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.502691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:36743 (local address 127.29.112.254:42151)
0812 18:59:08.502856 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:08.502861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:08.503732 (+ 871us) server_negotiation.cc:407] Connection header received
0812 18:59:08.505093 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:08.505096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:08.505165 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:08.505311 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:08.506631 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.507111 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.508324 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.508531 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.508603 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.511294 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:08.511297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:08.511300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:08.511324 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:08.512771 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:08.513298 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:08.513301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:08.513303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:08.513386 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:08.513720 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:08.513725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:08.513727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:08.513955 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:08.514057 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:08.514594 (+ 537us) server_negotiation.cc:299] Negotiation successful
0812 18:59:08.514711 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 18:59:08.515695 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.503117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:42151 (local address 127.29.112.193:36743)
0812 18:59:08.503578 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:08.503625 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:08.504896 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:08.505485 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:08.505492 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:08.505914 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:08.506440 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.506462 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.507235 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.507241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.508217 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.508227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.508671 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.508677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.508919 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.509478 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:08.509503 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:08.511142 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:08.512898 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:08.512903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:08.512915 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:08.513185 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:08.513481 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:08.513484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:08.513486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:08.513629 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:08.514057 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:08.514063 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:08.514324 (+ 261us) client_negotiation.cc:769] Sending connection context
0812 18:59:08.514532 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 18:59:08.514831 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":128,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
I20260812 18:59:08.517093 2837 heartbeater.cc:347] Connected to a master server at 127.29.112.254:42151
I20260812 18:59:08.517355 2837 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:08.517858 2837 heartbeater.cc:511] Master 127.29.112.254:42151 requested a full tablet report, sending...
I20260812 18:59:08.519430 2612 ts_manager.cc:194] Registered new tserver with Master: 6f9a02345d674eeb8a617294ba4e73f1 (127.29.112.193:45799)
I20260812 18:59:08.520511 2612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:36743
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 18:59:08.532886 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:42151
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:33495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:08.644490 2843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:08.644742 2843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:08.644845 2843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:08.648545 2843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:08.648615 2843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:08.648690 2843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:08.653419 2843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:33495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:08.654429 2843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:08.655383 2843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:08.664667 2848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:08.664690 2849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:08.664687 2851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:08.664925 2843 server_base.cc:1034] running on GCE node
I20260812 18:59:08.665431 2843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:08.666085 2843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:08.667243 2843 hybrid_clock.cc:648] HybridClock initialized: now 1786561148667218 us; error 38 us; skew 500 ppm
Aug 12 18:59:08 dist-test-slave-6nmv krb5kdc[2564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:08.670037 2843 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 18:59:08.670746 2843 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:08.670804 2843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:08.671005 2843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:08.672962 2843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "464f2719d7b94a3990d81595202c45f7"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "97f9c41d19ceb36c13f09bd929222df2"
server_key_iv: "0669374fd2f90f7ba0b6ec173513f228"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.673482 2843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "464f2719d7b94a3990d81595202c45f7"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "97f9c41d19ceb36c13f09bd929222df2"
server_key_iv: "0669374fd2f90f7ba0b6ec173513f228"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.676923 2843 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 18:59:08.679224 2861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:08.680370 2843 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:08.680539 2843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "464f2719d7b94a3990d81595202c45f7"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "97f9c41d19ceb36c13f09bd929222df2"
server_key_iv: "0669374fd2f90f7ba0b6ec173513f228"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.680642 2843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:08.690587 2843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:08.693390 2843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:08.693601 2843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:08.694187 2843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:08.695077 2843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:08.695165 2843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:08.695254 2843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:08.695302 2843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:08.704449 2843 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:36989
I20260812 18:59:08.704478 2994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:36989 every 8 connection(s)
I20260812 18:59:08.705091 2843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:08.709098 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 2843
I20260812 18:59:08.709244 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:08.709538 30147 external_mini_cluster.cc:1428] Setting key bdd3ee3733e4994639dab1f3030807d8
Aug 12 18:59:08 dist-test-slave-6nmv krb5kdc[2564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:08.719700 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.706858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:53473 (local address 127.29.112.254:42151)
0812 18:59:08.707009 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:08.707013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:08.707772 (+ 759us) server_negotiation.cc:407] Connection header received
0812 18:59:08.708994 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:08.708997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:08.709060 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:08.709156 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:08.710431 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.711035 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.712173 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.712395 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.712510 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.715353 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:08.715359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:08.715365 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:08.715398 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:08.717354 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:08.717986 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:08.717992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:08.717997 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:08.718105 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:08.718504 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:08.718507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:08.718510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:08.718743 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:08.718856 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:08.719376 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 18:59:08.719502 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 18:59:08.720399 2998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.707214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:42151 (local address 127.29.112.194:53473)
0812 18:59:08.707635 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:08.707672 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:08.708799 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:08.709331 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:08.709341 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:08.709736 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:08.710288 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.710308 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.711159 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.711162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.712045 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.712054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.712549 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.712553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.712714 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.713368 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:08.713395 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:08.715156 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:08.717510 (+ 2354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:08.717528 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:08.717538 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:08.717805 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:08.718219 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:08.718221 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:08.718222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:08.718383 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:08.718876 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:08.718883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:08.719104 (+ 221us) client_negotiation.cc:769] Sending connection context
0812 18:59:08.719269 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 18:59:08.719519 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
I20260812 18:59:08.721537 2995 heartbeater.cc:347] Connected to a master server at 127.29.112.254:42151
I20260812 18:59:08.721779 2995 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:08.722239 2995 heartbeater.cc:511] Master 127.29.112.254:42151 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
I20260812 18:59:08.723383 2612 ts_manager.cc:194] Registered new tserver with Master: 464f2719d7b94a3990d81595202c45f7 (127.29.112.194:36989)
I20260812 18:59:08.724177 2612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:53473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 18:59:08.741698 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:42151
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:33495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:08.851846 3003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:08.852073 3003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:08.852115 3003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:08.855947 3003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:08.855993 3003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:08.856048 3003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:08.860948 3003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:33495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:08.861927 3003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:08.862819 3003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:08.871863 3009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:08.871935 3012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:08.872047 3003 server_base.cc:1034] running on GCE node
W20260812 18:59:08.871881 3010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:08.872625 3003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:08.873296 3003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:08.874521 3003 hybrid_clock.cc:648] HybridClock initialized: now 1786561148874482 us; error 33 us; skew 500 ppm
Aug 12 18:59:08 dist-test-slave-6nmv krb5kdc[2564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:08.877388 3003 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 18:59:08.878130 3003 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:08.878223 3003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:08.878443 3003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:08.880373 3003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "310bcf58b8654370b71bde8c6c213277"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "82a3a16ffd854830eb62abd4d88b629d"
server_key_iv: "706254969ed078fbd5059276a620c1e4"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.880925 3003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "310bcf58b8654370b71bde8c6c213277"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "82a3a16ffd854830eb62abd4d88b629d"
server_key_iv: "706254969ed078fbd5059276a620c1e4"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.884446 3003 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 18:59:08.886790 3020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:08.887972 3003 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:08.888114 3003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "310bcf58b8654370b71bde8c6c213277"
format_stamp: "Formatted at 2026-08-12 18:59:08 on dist-test-slave-6nmv"
server_key: "82a3a16ffd854830eb62abd4d88b629d"
server_key_iv: "706254969ed078fbd5059276a620c1e4"
server_key_version: "encryptionkey@0"
I20260812 18:59:08.888222 3003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:08.905234 3003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:08.908257 3003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:08.908427 3003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:08.908936 3003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:08.909637 3003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:08.909699 3003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:08.909763 3003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:08.909804 3003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:08.918320 3003 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:40609
I20260812 18:59:08.918344 3156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:40609 every 8 connection(s)
I20260812 18:59:08.918859 3003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
Aug 12 18:59:08 dist-test-slave-6nmv krb5kdc[2564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:08.928371 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 3003
I20260812 18:59:08.928503 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:08.928774 30147 external_mini_cluster.cc:1428] Setting key a8898b45d7af621ac14881fef2a148b7
I20260812 18:59:08.935304 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.920512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:44069 (local address 127.29.112.254:42151)
0812 18:59:08.920635 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:08.920639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:08.921368 (+ 729us) server_negotiation.cc:407] Connection header received
0812 18:59:08.922709 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:08.922712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:08.922755 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:08.922825 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:08.923870 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.924369 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.925349 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.925550 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.925623 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.927973 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:08.927976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:08.927978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:08.927999 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:08.933576 (+ 5577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:08.934070 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:08.934072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:08.934073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:08.934144 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:08.934432 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:08.934434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:08.934435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:08.934596 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:08.934660 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:08.935087 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 18:59:08.935175 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 18:59:08.936034 3159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.920841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:42151 (local address 127.29.112.195:44069)
0812 18:59:08.921230 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:08.921267 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:08.922522 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:08.922969 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:08.922974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:08.923280 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:08.923735 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.923765 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.924465 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.924467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.925244 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.925252 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.925673 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.925675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.925855 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.926365 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:08.926380 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:08.927817 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:08.933673 (+ 5856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:08.933678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:08.933687 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:08.933990 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:08.934221 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:08.934223 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:08.934224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:08.934356 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:08.934669 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:08.934673 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:08.934878 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 18:59:08.935027 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 18:59:08.935268 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":102,"threads_started":1}
I20260812 18:59:08.937032 3157 heartbeater.cc:347] Connected to a master server at 127.29.112.254:42151
I20260812 18:59:08.937245 3157 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:08.937615 3157 heartbeater.cc:511] Master 127.29.112.254:42151 requested a full tablet report, sending...
I20260812 18:59:08.938511 2612 ts_manager.cc:194] Registered new tserver with Master: 310bcf58b8654370b71bde8c6c213277 (127.29.112.195:40609)
I20260812 18:59:08.939059 2612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:44069
I20260812 18:59:08.942283 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 18:59:08.950227 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.944056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33862 (local address 127.29.112.254:42151)
0812 18:59:08.944157 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:08.944160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:08.944304 (+ 144us) server_negotiation.cc:407] Connection header received
0812 18:59:08.944392 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:08.944393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:08.944425 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:08.944475 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:08.945059 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.945530 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.946375 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.946587 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.946657 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.947787 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:08.947789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:08.947791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:08.947805 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:08.948899 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:08.949243 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:08.949245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:08.949246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:08.949311 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:08.949514 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:08.949516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:08.949517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:08.949686 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:08.949777 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:08.950053 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 18:59:08.950133 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 18:59:08.952659 2612 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:59:08.955727 2612 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:59:08.969340 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.964205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56670 (local address 127.29.112.194:36989)
0812 18:59:08.964815 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:08.964827 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:08.964885 (+ 58us) server_negotiation.cc:407] Connection header received
0812 18:59:08.965024 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:08.965032 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:08.965242 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:08.965702 (+ 460us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:08.965873 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.966934 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.968272 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.968736 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.968858 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.968976 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:08.969101 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 18:59:08.969187 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":128,"threads_started":1}
I20260812 18:59:08.969378 3169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.964032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:36989 (local address 127.0.0.1:56670)
0812 18:59:08.964803 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:08.964817 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:08.964912 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:08.965440 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:08.965443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:08.965462 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:08.965556 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.965562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.967112 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.967115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.968171 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.968177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.968851 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.968855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.968948 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.968981 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 18:59:08.969079 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 18:59:08.969221 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":27,"thread_start_us":99,"threads_started":1}
I20260812 18:59:08.969391 3173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.963332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:40609 (local address 127.0.0.1:37718)
0812 18:59:08.964367 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:08.964380 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:08.964483 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:08.965513 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:08.965516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:08.965534 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:08.965644 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.965653 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.966989 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.966992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.967949 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.967957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.968946 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.968949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.969047 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.969066 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 18:59:08.969113 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 18:59:08.969236 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":49,"thread_start_us":103,"threads_started":1}
I20260812 18:59:08.969899 3168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.963238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45799 (local address 127.0.0.1:45584)
0812 18:59:08.963502 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:08.963542 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:08.963647 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:08.964856 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:08.964862 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:08.964888 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:08.965034 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.965042 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.966553 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.966557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.967994 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:08.968004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.968946 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.968949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:08.969231 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.969247 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 18:59:08.969313 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 18:59:08.969364 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":318,"spinlock_wait_cycles":896,"thread_start_us":57,"threads_started":1}
I20260812 18:59:08.970094 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.963910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37718 (local address 127.29.112.195:40609)
0812 18:59:08.965151 (+ 1241us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:08.965164 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:08.965190 (+ 26us) server_negotiation.cc:407] Connection header received
0812 18:59:08.965242 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:08.965246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:08.965387 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:08.965533 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:08.965767 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.966815 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.968090 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.968811 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.969587 (+ 776us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.969711 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:08.969778 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 18:59:08.969888 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1141,"thread_start_us":183,"threads_started":1}
I20260812 18:59:08.970507 3170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:08.963355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45584 (local address 127.29.112.193:45799)
0812 18:59:08.963931 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:08.963939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:08.963964 (+ 25us) server_negotiation.cc:407] Connection header received
0812 18:59:08.964454 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:08.964461 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:08.964682 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:08.964888 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:08.965151 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.966389 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.968111 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:08.968811 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:08.969004 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:08.969114 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:08.970259 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0812 18:59:08.970370 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":94,"threads_started":1}
I20260812 18:59:08.972137 2920 tablet_service.cc:1467] Processing CreateTablet for tablet 87c5d6e7f5a54602b604b218470197ca (DEFAULT_TABLE table=test-table [id=09ffbe19356a41a1b5d3cdf002c5888a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:08.972604 3077 tablet_service.cc:1467] Processing CreateTablet for tablet 87c5d6e7f5a54602b604b218470197ca (DEFAULT_TABLE table=test-table [id=09ffbe19356a41a1b5d3cdf002c5888a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:08.973083 2920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c5d6e7f5a54602b604b218470197ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:08.972846 2765 tablet_service.cc:1467] Processing CreateTablet for tablet 87c5d6e7f5a54602b604b218470197ca (DEFAULT_TABLE table=test-table [id=09ffbe19356a41a1b5d3cdf002c5888a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:08.973582 3077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c5d6e7f5a54602b604b218470197ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:08.973666 2765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c5d6e7f5a54602b604b218470197ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:08.978786 3175 tablet_bootstrap.cc:492] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: Bootstrap starting.
I20260812 18:59:08.979954 3176 tablet_bootstrap.cc:492] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: Bootstrap starting.
I20260812 18:59:08.981899 3176 tablet_bootstrap.cc:654] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:08.981899 3175 tablet_bootstrap.cc:654] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:08.982146 3177 tablet_bootstrap.cc:492] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1: Bootstrap starting.
I20260812 18:59:08.983258 3175 log.cc:824] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:08.983258 3176 log.cc:824] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:08.983948 3177 tablet_bootstrap.cc:654] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:08.985149 3177 log.cc:824] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:08.985334 3176 tablet_bootstrap.cc:492] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: No bootstrap required, opened a new log
I20260812 18:59:08.985532 3175 tablet_bootstrap.cc:492] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: No bootstrap required, opened a new log
I20260812 18:59:08.985522 3176 ts_tablet_manager.cc:1397] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260812 18:59:08.985759 3175 ts_tablet_manager.cc:1397] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260812 18:59:08.986779 3177 tablet_bootstrap.cc:492] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1: No bootstrap required, opened a new log
I20260812 18:59:08.986968 3177 ts_tablet_manager.cc:1397] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 18:59:08.989351 3176 raft_consensus.cc:348] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9a02345d674eeb8a617294ba4e73f1" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45799 } } peers { permanent_uuid: "310bcf58b8654370b71bde8c6c213277" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40609 } } peers { permanent_uuid: "464f2719d7b94a3990d81595202c45f7" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 36989 } }
I20260812 18:59:08.989351 3175 raft_consensus.cc:348] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9a02345d674eeb8a617294ba4e73f1" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45799 } } peers { permanent_uuid: "310bcf58b8654370b71bde8c6c213277" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40609 } } peers { permanent_uuid: "464f2719d7b94a3990d81595202c45f7" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 36989 } }
I20260812 18:59:08.989605 3176 raft_consensus.cc:374] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:08.989605 3175 raft_consensus.cc:374] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:08.989485 3177 raft_consensus.cc:348] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9a02345d674eeb8a617294ba4e73f1" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45799 } } peers { permanent_uuid: "310bcf58b8654370b71bde8c6c213277" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40609 } } peers { permanent_uuid: "464f2719d7b94a3990d81595202c45f7" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 36989 } }
I20260812 18:59:08.989671 3175 raft_consensus.cc:729] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464f2719d7b94a3990d81595202c45f7, State: Initialized, Role: FOLLOWER
I20260812 18:59:08.989671 3176 raft_consensus.cc:729] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 310bcf58b8654370b71bde8c6c213277, State: Initialized, Role: FOLLOWER
I20260812 18:59:08.989692 3177 raft_consensus.cc:374] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:08.989743 3177 raft_consensus.cc:729] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9a02345d674eeb8a617294ba4e73f1, State: Initialized, Role: FOLLOWER
I20260812 18:59:08.990134 3177 consensus_queue.cc:260] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9a02345d674eeb8a617294ba4e73f1" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45799 } } peers { permanent_uuid: "310bcf58b8654370b71bde8c6c213277" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40609 } } peers { permanent_uuid: "464f2719d7b94a3990d81595202c45f7" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 36989 } }
I20260812 18:59:08.990130 3176 consensus_queue.cc:260] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9a02345d674eeb8a617294ba4e73f1" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45799 } } peers { permanent_uuid: "310bcf58b8654370b71bde8c6c213277" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40609 } } peers { permanent_uuid: "464f2719d7b94a3990d81595202c45f7" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 36989 } }
I20260812 18:59:08.990130 3175 consensus_queue.cc:260] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9a02345d674eeb8a617294ba4e73f1" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45799 } } peers { permanent_uuid: "310bcf58b8654370b71bde8c6c213277" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40609 } } peers { permanent_uuid: "464f2719d7b94a3990d81595202c45f7" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 36989 } }
I20260812 18:59:08.990746 2995 heartbeater.cc:503] Master 127.29.112.254:42151 was elected leader, sending a full tablet report...
I20260812 18:59:08.990981 3157 heartbeater.cc:503] Master 127.29.112.254:42151 was elected leader, sending a full tablet report...
I20260812 18:59:08.990993 3175 ts_tablet_manager.cc:1428] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260812 18:59:08.991643 2837 heartbeater.cc:503] Master 127.29.112.254:42151 was elected leader, sending a full tablet report...
I20260812 18:59:08.991956 3177 ts_tablet_manager.cc:1428] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 18:59:08.992053 3176 ts_tablet_manager.cc:1428] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260812 18:59:09.002101 2838 tablet.cc:2369] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 18:59:09.169967 3158 tablet.cc:2369] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:59:09.193065 3181 raft_consensus.cc:482] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:59:09.193311 3181 raft_consensus.cc:504] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9a02345d674eeb8a617294ba4e73f1" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45799 } } peers { permanent_uuid: "310bcf58b8654370b71bde8c6c213277" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40609 } } peers { permanent_uuid: "464f2719d7b94a3990d81595202c45f7" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 36989 } }
I20260812 18:59:09.194602 3181 leader_election.cc:288] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 310bcf58b8654370b71bde8c6c213277 (127.29.112.195:40609), 464f2719d7b94a3990d81595202c45f7 (127.29.112.194:36989)
I20260812 18:59:09.198577 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:09.194827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:36989 (local address 127.29.112.193:53453)
0812 18:59:09.195159 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:09.195172 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:09.195271 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:09.195686 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:09.195691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:09.195707 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:09.195815 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:09.195819 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:09.196581 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:09.196588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:09.197428 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:09.197436 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:09.198123 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:09.198127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:09.198218 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:09.198247 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 18:59:09.198338 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 18:59:09.198409 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":48,"thread_start_us":96,"threads_started":1}
I20260812 18:59:09.198611 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:09.195318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:53453 (local address 127.29.112.194:36989)
0812 18:59:09.195451 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:09.195457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:09.195471 (+ 14us) server_negotiation.cc:407] Connection header received
0812 18:59:09.195523 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:09.195525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:09.195561 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:09.195656 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:09.195938 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:09.196445 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:09.197554 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:09.198014 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:09.198122 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:09.198172 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:09.198365 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 18:59:09.198502 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 18:59:09.198711 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:09.194957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:36123 (local address 127.29.112.195:40609)
0812 18:59:09.195155 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:09.195161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:09.195354 (+ 193us) server_negotiation.cc:407] Connection header received
0812 18:59:09.195484 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:09.195486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:09.195528 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:09.195589 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:09.195931 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:09.196440 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:09.197573 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:09.198035 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:09.198157 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:09.198209 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:09.198487 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 18:59:09.198580 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 18:59:09.198627 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:09.194840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:40609 (local address 127.29.112.193:36123)
0812 18:59:09.195206 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:09.195221 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:09.195360 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:09.195629 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:09.195634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:09.195652 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:09.195808 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:09.195813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:09.196558 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:09.196564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:09.197438 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:09.197445 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:09.198162 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:09.198165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:09.198261 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:09.198277 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 18:59:09.198429 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 18:59:09.198497 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":25,"thread_start_us":132,"threads_started":1}
I20260812 18:59:09.199285 2945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c5d6e7f5a54602b604b218470197ca" candidate_uuid: "6f9a02345d674eeb8a617294ba4e73f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464f2719d7b94a3990d81595202c45f7" is_pre_election: true
I20260812 18:59:09.199373 3101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c5d6e7f5a54602b604b218470197ca" candidate_uuid: "6f9a02345d674eeb8a617294ba4e73f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "310bcf58b8654370b71bde8c6c213277" is_pre_election: true
I20260812 18:59:09.199579 2945 raft_consensus.cc:2454] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f9a02345d674eeb8a617294ba4e73f1 in term 0.
I20260812 18:59:09.199690 3101 raft_consensus.cc:2454] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f9a02345d674eeb8a617294ba4e73f1 in term 0.
I20260812 18:59:09.200054 2701 leader_election.cc:302] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464f2719d7b94a3990d81595202c45f7, 6f9a02345d674eeb8a617294ba4e73f1; no voters:
I20260812 18:59:09.200286 3181 raft_consensus.cc:2781] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:59:09.200373 3181 raft_consensus.cc:482] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:59:09.200426 3181 raft_consensus.cc:3037] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:09.201488 3181 raft_consensus.cc:504] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9a02345d674eeb8a617294ba4e73f1" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45799 } } peers { permanent_uuid: "310bcf58b8654370b71bde8c6c213277" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40609 } } peers { permanent_uuid: "464f2719d7b94a3990d81595202c45f7" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 36989 } }
I20260812 18:59:09.201858 3181 leader_election.cc:288] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [CANDIDATE]: Term 1 election: Requested vote from peers 310bcf58b8654370b71bde8c6c213277 (127.29.112.195:40609), 464f2719d7b94a3990d81595202c45f7 (127.29.112.194:36989)
I20260812 18:59:09.202243 3101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c5d6e7f5a54602b604b218470197ca" candidate_uuid: "6f9a02345d674eeb8a617294ba4e73f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "310bcf58b8654370b71bde8c6c213277"
I20260812 18:59:09.202358 2945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c5d6e7f5a54602b604b218470197ca" candidate_uuid: "6f9a02345d674eeb8a617294ba4e73f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464f2719d7b94a3990d81595202c45f7"
I20260812 18:59:09.202412 3101 raft_consensus.cc:3037] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:09.202533 2945 raft_consensus.cc:3037] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:09.203568 2945 raft_consensus.cc:2454] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9a02345d674eeb8a617294ba4e73f1 in term 1.
I20260812 18:59:09.203823 3101 raft_consensus.cc:2454] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f9a02345d674eeb8a617294ba4e73f1 in term 1.
I20260812 18:59:09.203851 2701 leader_election.cc:302] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464f2719d7b94a3990d81595202c45f7, 6f9a02345d674eeb8a617294ba4e73f1; no voters:
I20260812 18:59:09.204052 3181 raft_consensus.cc:2781] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:09.204308 3181 raft_consensus.cc:686] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 1 LEADER]: Becoming Leader. State: Replica: 6f9a02345d674eeb8a617294ba4e73f1, State: Running, Role: LEADER
I20260812 18:59:09.204672 3181 consensus_queue.cc:237] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9a02345d674eeb8a617294ba4e73f1" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45799 } } peers { permanent_uuid: "310bcf58b8654370b71bde8c6c213277" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40609 } } peers { permanent_uuid: "464f2719d7b94a3990d81595202c45f7" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 36989 } }
W20260812 18:59:09.206154 2997 tablet.cc:2369] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:59:09.208114 2611 catalog_manager.cc:5495] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f9a02345d674eeb8a617294ba4e73f1 (127.29.112.193). New cstate: current_term: 1 leader_uuid: "6f9a02345d674eeb8a617294ba4e73f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9a02345d674eeb8a617294ba4e73f1" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "310bcf58b8654370b71bde8c6c213277" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "464f2719d7b94a3990d81595202c45f7" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 36989 } health_report { overall_health: UNKNOWN } } }
I20260812 18:59:09.258354 3170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:09.254833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45600 (local address 127.29.112.193:45799)
0812 18:59:09.254968 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:09.254972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:09.254985 (+ 13us) server_negotiation.cc:407] Connection header received
0812 18:59:09.255151 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:09.255153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:09.255202 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:09.255335 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:59:09.255592 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:09.256087 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:09.256933 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:09.257148 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:09.257254 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:09.257570 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:59:09.257971 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:59:09.258099 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:09.258168 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 18:59:09.258219 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 18:59:09.263706 2945 raft_consensus.cc:1261] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7 [term 1 FOLLOWER]: Refusing update from remote peer 6f9a02345d674eeb8a617294ba4e73f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:09.263733 3101 raft_consensus.cc:1261] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277 [term 1 FOLLOWER]: Refusing update from remote peer 6f9a02345d674eeb8a617294ba4e73f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:09.264370 3181 consensus_queue.cc:1035] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "464f2719d7b94a3990d81595202c45f7" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 36989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:09.264602 3187 consensus_queue.cc:1035] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "310bcf58b8654370b71bde8c6c213277" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:09.274521 3191 mvcc.cc:205] Tried to move back new op lower bound from 7317754467377721344 to 7317754467145203712. Current Snapshot: MvccSnapshot[applied={T|T < 7317754467377721344 or (T in {7317754467377721344})}]
I20260812 18:59:09.281327 2610 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:09.281494 2610 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:09.283941 2610 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5052a77dd322433685cd626a20ff4b57: Sending DeleteTablet for 3 replicas of tablet 87c5d6e7f5a54602b604b218470197ca
I20260812 18:59:09.284647 2765 tablet_service.cc:1514] Processing DeleteTablet for tablet 87c5d6e7f5a54602b604b218470197ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:09 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:45584
I20260812 18:59:09.284724 3077 tablet_service.cc:1514] Processing DeleteTablet for tablet 87c5d6e7f5a54602b604b218470197ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:09 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:37718
I20260812 18:59:09.284822 2920 tablet_service.cc:1514] Processing DeleteTablet for tablet 87c5d6e7f5a54602b604b218470197ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:09 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:56670
I20260812 18:59:09.285130 3200 tablet_replica.cc:330] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1: stopping tablet replica
I20260812 18:59:09.285449 3200 raft_consensus.cc:2229] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:59:09.285588 3201 tablet_replica.cc:330] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: stopping tablet replica
I20260812 18:59:09.285687 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 2680
I20260812 18:59:09.285799 3200 raft_consensus.cc:2258] T 87c5d6e7f5a54602b604b218470197ca P 6f9a02345d674eeb8a617294ba4e73f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:09.288070 3201 raft_consensus.cc:2229] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:09.288411 3201 raft_consensus.cc:2258] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:09.290405 3201 ts_tablet_manager.cc:1905] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:09.290616 3202 tablet_replica.cc:330] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: stopping tablet replica
I20260812 18:59:09.290872 3202 raft_consensus.cc:2229] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:09.291121 3202 raft_consensus.cc:2258] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:09.292764 3202 ts_tablet_manager.cc:1905] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:09.293228 3201 ts_tablet_manager.cc:1918] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:09.293313 3201 log.cc:1196] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/87c5d6e7f5a54602b604b218470197ca
I20260812 18:59:09.293617 3201 ts_tablet_manager.cc:1939] T 87c5d6e7f5a54602b604b218470197ca P 310bcf58b8654370b71bde8c6c213277: Deleting consensus metadata
I20260812 18:59:09.294746 2596 catalog_manager.cc:4841] TS 310bcf58b8654370b71bde8c6c213277 (127.29.112.195:40609): tablet 87c5d6e7f5a54602b604b218470197ca (table test-table [id=09ffbe19356a41a1b5d3cdf002c5888a]) successfully deleted
I20260812 18:59:09.295274 3202 ts_tablet_manager.cc:1918] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:09.295362 3202 log.cc:1196] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/87c5d6e7f5a54602b604b218470197ca
I20260812 18:59:09.295639 3202 ts_tablet_manager.cc:1939] T 87c5d6e7f5a54602b604b218470197ca P 464f2719d7b94a3990d81595202c45f7: Deleting consensus metadata
I20260812 18:59:09.296582 2599 catalog_manager.cc:4841] TS 464f2719d7b94a3990d81595202c45f7 (127.29.112.194:36989): tablet 87c5d6e7f5a54602b604b218470197ca (table test-table [id=09ffbe19356a41a1b5d3cdf002c5888a]) successfully deleted
W20260812 18:59:09.297309 2597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:45799 (error 108)
I20260812 18:59:09.298038 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 2843
I20260812 18:59:09.298291 3168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:09.297994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45799 (local address 127.0.0.1:45606)
0812 18:59:09.298138 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:09.298207 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:45799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 18:59:09.299021 2597 catalog_manager.cc:4584] TS 6f9a02345d674eeb8a617294ba4e73f1 (127.29.112.193:45799): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 87c5d6e7f5a54602b604b218470197ca: Network error: Client connection negotiation failed: client connection to 127.29.112.193:45799: connect: Connection refused (error 111)
I20260812 18:59:09.305330 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 3003
I20260812 18:59:09.312546 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 2578
2026-08-12T18:59:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2482 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T18:59:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:09Z Disabled control of system clock
I20260812 18:59:09.344412 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:44267
--webserver_interface=127.29.112.254
--webserver_port=0
--builtin_ntp_servers=127.29.112.212:34345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:44267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 18:59:09.445518 3207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:09.445730 3207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:09.445765 3207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:09.449888 3207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:09.449932 3207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:09.449946 3207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:09.449961 3207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:09.449975 3207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:09.454499 3207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:44267
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:44267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:09.455374 3207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:09.456617 3207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:09.464680 3213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:09.464685 3212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:09.464840 3207 server_base.cc:1034] running on GCE node
W20260812 18:59:09.464764 3216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:09.465425 3207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:09.466470 3207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:09.467643 3207 hybrid_clock.cc:648] HybridClock initialized: now 1786561149467631 us; error 33 us; skew 500 ppm
I20260812 18:59:09.469437 3207 webserver.cc:460] Webserver started at http://127.29.112.254:40885/ using document root <none> and password file <none>
I20260812 18:59:09.470033 3207 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:09.470096 3207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:09.470269 3207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:09.472112 3207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "6ec64504252f4c0590b677c162f2e40a"
format_stamp: "Formatted at 2026-08-12 18:59:09 on dist-test-slave-6nmv"
server_key: "c2e5691a0c487cca970584b224a433e3"
server_key_iv: "90a95adbe3687cf2b331fdcb3f6f1fc2"
server_key_version: "encryptionkey@0"
I20260812 18:59:09.472600 3207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "6ec64504252f4c0590b677c162f2e40a"
format_stamp: "Formatted at 2026-08-12 18:59:09 on dist-test-slave-6nmv"
server_key: "c2e5691a0c487cca970584b224a433e3"
server_key_iv: "90a95adbe3687cf2b331fdcb3f6f1fc2"
server_key_version: "encryptionkey@0"
I20260812 18:59:09.476094 3207 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 18:59:09.478637 3222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:09.479871 3207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 18:59:09.479974 3207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "6ec64504252f4c0590b677c162f2e40a"
format_stamp: "Formatted at 2026-08-12 18:59:09 on dist-test-slave-6nmv"
server_key: "c2e5691a0c487cca970584b224a433e3"
server_key_iv: "90a95adbe3687cf2b331fdcb3f6f1fc2"
server_key_version: "encryptionkey@0"
I20260812 18:59:09.480074 3207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:09.484218 3207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:09.484735 3207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:09.484863 3207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:09.491235 3207 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:44267
I20260812 18:59:09.491243 3284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:44267 every 8 connection(s)
I20260812 18:59:09.492101 3207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:09.495163 3285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:09.499568 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 3207
I20260812 18:59:09.499696 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:09.499980 30147 external_mini_cluster.cc:1428] Setting key e8cf4330266256e0bd2fae980e8e19c9
I20260812 18:59:09.500615 3285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a: Bootstrap starting.
I20260812 18:59:09.502861 3285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:09.503770 3285 log.cc:824] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:09.505188 3290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:09.501255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44386 (local address 127.29.112.254:44267)
0812 18:59:09.501788 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:09.501802 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:09.501880 (+ 78us) server_negotiation.cc:407] Connection header received
0812 18:59:09.502487 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:09.502502 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:09.502819 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:09.503130 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:09.504009 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:09.504013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:09.504024 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:09.504034 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:09.504097 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:09.504130 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:09.504279 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:09.504414 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 18:59:09.504626 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":103,"threads_started":1}
I20260812 18:59:09.505724 3285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a: No bootstrap required, opened a new log
I20260812 18:59:09.508770 3285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec64504252f4c0590b677c162f2e40a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44267 } }
I20260812 18:59:09.508957 3285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:09.509032 3285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ec64504252f4c0590b677c162f2e40a, State: Initialized, Role: FOLLOWER
I20260812 18:59:09.509428 3285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec64504252f4c0590b677c162f2e40a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44267 } }
I20260812 18:59:09.509563 3285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:09.509619 3285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:09.509720 3285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:09.510456 3285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec64504252f4c0590b677c162f2e40a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44267 } }
I20260812 18:59:09.510794 3285 leader_election.cc:302] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ec64504252f4c0590b677c162f2e40a; no voters:
I20260812 18:59:09.511075 3285 leader_election.cc:288] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:09.511154 3292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:09.511420 3292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [term 1 LEADER]: Becoming Leader. State: Replica: 6ec64504252f4c0590b677c162f2e40a, State: Running, Role: LEADER
I20260812 18:59:09.511744 3292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec64504252f4c0590b677c162f2e40a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44267 } }
I20260812 18:59:09.512252 3285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:09.513312 3294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ec64504252f4c0590b677c162f2e40a. Latest consensus state: current_term: 1 leader_uuid: "6ec64504252f4c0590b677c162f2e40a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec64504252f4c0590b677c162f2e40a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44267 } } }
I20260812 18:59:09.513435 3294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:09.513662 3293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ec64504252f4c0590b677c162f2e40a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ec64504252f4c0590b677c162f2e40a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44267 } } }
I20260812 18:59:09.513759 3293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:09.514401 3302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:09.516721 3302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:09.521457 3302 catalog_manager.cc:1347] Generated new cluster ID: 460ccb07f8e94b83891caf22dded9a53
I20260812 18:59:09.521510 3302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:09.540316 3302 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:09.540678 3302 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:09.550326 3302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a: Generated new TSK 0
I20260812 18:59:09.550899 3302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 18:59:09.562650 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44267
--builtin_ntp_servers=127.29.112.212:34345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 18:59:09.668931 3314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:09.669145 3314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:09.669181 3314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:09.672781 3314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:09.672824 3314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:09.672887 3314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:09.677215 3314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:09.678020 3314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:09.679294 3314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:09.688512 3321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:09.688577 3314 server_base.cc:1034] running on GCE node
W20260812 18:59:09.688592 3322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:09.688524 3324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:09.689110 3314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:09.689669 3314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:09.690827 3314 hybrid_clock.cc:648] HybridClock initialized: now 1786561149690793 us; error 60 us; skew 500 ppm
I20260812 18:59:09.692508 3314 webserver.cc:460] Webserver started at http://127.29.112.193:45711/ using document root <none> and password file <none>
I20260812 18:59:09.693048 3314 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:09.693120 3314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:09.693346 3314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:09.695211 3314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "ff6755591b5648cbb6b15fda540262d9"
format_stamp: "Formatted at 2026-08-12 18:59:09 on dist-test-slave-6nmv"
server_key: "ef2b0e602269eba147549e6e872aaa2a"
server_key_iv: "5c8ba59e362a0c2a176ffe49af2405cb"
server_key_version: "encryptionkey@0"
I20260812 18:59:09.695773 3314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "ff6755591b5648cbb6b15fda540262d9"
format_stamp: "Formatted at 2026-08-12 18:59:09 on dist-test-slave-6nmv"
server_key: "ef2b0e602269eba147549e6e872aaa2a"
server_key_iv: "5c8ba59e362a0c2a176ffe49af2405cb"
server_key_version: "encryptionkey@0"
I20260812 18:59:09.699162 3314 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 18:59:09.701417 3330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:09.702610 3314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:09.702744 3314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "ff6755591b5648cbb6b15fda540262d9"
format_stamp: "Formatted at 2026-08-12 18:59:09 on dist-test-slave-6nmv"
server_key: "ef2b0e602269eba147549e6e872aaa2a"
server_key_iv: "5c8ba59e362a0c2a176ffe49af2405cb"
server_key_version: "encryptionkey@0"
I20260812 18:59:09.702869 3314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:09.707028 3314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:09.707585 3314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:09.707758 3314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:09.708230 3314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:09.709050 3314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:09.709111 3314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:09.709168 3314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:09.709214 3314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:09.717535 3314 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:42347
I20260812 18:59:09.717568 3459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:42347 every 8 connection(s)
I20260812 18:59:09.718410 3314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:09.719219 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 3314
I20260812 18:59:09.719359 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:09.719622 30147 external_mini_cluster.cc:1428] Setting key c501244a0843c18b6d7eb444ad008000
I20260812 18:59:09.721549 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44267
--builtin_ntp_servers=127.29.112.212:34345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 18:59:09.724339 3290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:09.720122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:48045 (local address 127.29.112.254:44267)
0812 18:59:09.720251 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:09.720256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:09.721327 (+ 1071us) server_negotiation.cc:407] Connection header received
0812 18:59:09.722759 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:09.722762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:09.722799 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:09.722870 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:09.723897 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:09.723900 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:09.723903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:09.723914 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:09.723934 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:09.723957 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:09.724040 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:09.724184 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 18:59:09.724223 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 18:59:09.725044 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:09.720516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44267 (local address 127.29.112.193:48045)
0812 18:59:09.721204 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:09.721239 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:09.722572 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:09.723017 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:09.723022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:09.723100 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:09.723655 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:09.723657 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:09.723690 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:09.723764 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:09.724039 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:09.724049 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:09.724057 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 18:59:09.724222 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 18:59:09.724425 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":162,"threads_started":1}
I20260812 18:59:09.726173 3460 heartbeater.cc:347] Connected to a master server at 127.29.112.254:44267
I20260812 18:59:09.726394 3460 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:09.726770 3460 heartbeater.cc:511] Master 127.29.112.254:44267 requested a full tablet report, sending...
I20260812 18:59:09.728119 3242 ts_manager.cc:194] Registered new tserver with Master: ff6755591b5648cbb6b15fda540262d9 (127.29.112.193:42347)
W20260812 18:59:09.827487 3465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:09.827742 3465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:09.827833 3465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:09.831727 3465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:09.831800 3465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:09.831903 3465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:09.836576 3465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:09.837431 3465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:09.838686 3465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:09.846987 3472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:09.847003 3471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:09.847007 3474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:09.847193 3465 server_base.cc:1034] running on GCE node
I20260812 18:59:09.847604 3465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:09.848163 3465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:09.849344 3465 hybrid_clock.cc:648] HybridClock initialized: now 1786561149849329 us; error 32 us; skew 500 ppm
I20260812 18:59:09.851044 3465 webserver.cc:460] Webserver started at http://127.29.112.194:46251/ using document root <none> and password file <none>
I20260812 18:59:09.851554 3465 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:09.851663 3465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:09.851926 3465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:09.853753 3465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "7ba980c52f7d4e09a06dc2eab7d69a19"
format_stamp: "Formatted at 2026-08-12 18:59:09 on dist-test-slave-6nmv"
server_key: "f9295ad78524597264636ce3fcd3d387"
server_key_iv: "27e2745fd6f80cd3710dfbaa79c0b527"
server_key_version: "encryptionkey@0"
I20260812 18:59:09.854413 3465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "7ba980c52f7d4e09a06dc2eab7d69a19"
format_stamp: "Formatted at 2026-08-12 18:59:09 on dist-test-slave-6nmv"
server_key: "f9295ad78524597264636ce3fcd3d387"
server_key_iv: "27e2745fd6f80cd3710dfbaa79c0b527"
server_key_version: "encryptionkey@0"
I20260812 18:59:09.857894 3465 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 18:59:09.860230 3480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:09.861330 3465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 18:59:09.861475 3465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "7ba980c52f7d4e09a06dc2eab7d69a19"
format_stamp: "Formatted at 2026-08-12 18:59:09 on dist-test-slave-6nmv"
server_key: "f9295ad78524597264636ce3fcd3d387"
server_key_iv: "27e2745fd6f80cd3710dfbaa79c0b527"
server_key_version: "encryptionkey@0"
I20260812 18:59:09.861590 3465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:09.865674 3465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:09.866286 3465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:09.866501 3465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:09.866994 3465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:09.867816 3465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:09.867892 3465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:09.867959 3465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:09.868011 3465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:09.876860 3465 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:41319
I20260812 18:59:09.876878 3612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:41319 every 8 connection(s)
I20260812 18:59:09.877619 3465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:09.879175 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 3465
I20260812 18:59:09.879308 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:09.879606 30147 external_mini_cluster.cc:1428] Setting key d30370fdaf0e73584e4946c9d6f9f9ad
I20260812 18:59:09.881776 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44267
--builtin_ntp_servers=127.29.112.212:34345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 18:59:09.884160 3290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:09.880072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:32813 (local address 127.29.112.254:44267)
0812 18:59:09.880237 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:09.880243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:09.880753 (+ 510us) server_negotiation.cc:407] Connection header received
0812 18:59:09.882167 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:09.882170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:09.882206 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:09.882268 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:09.883527 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:09.883533 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:09.883538 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:09.883555 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:09.883586 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:09.883620 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:09.883735 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:09.883881 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 18:59:09.883929 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 18:59:09.884939 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:09.879872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44267 (local address 127.29.112.194:32813)
0812 18:59:09.880604 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:09.880649 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:09.881956 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:09.882437 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:09.882446 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:09.882552 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:09.883214 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:09.883218 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:09.883268 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:09.883349 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:09.883735 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:09.883742 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:09.883746 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 18:59:09.883882 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 18:59:09.884130 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":138,"threads_started":1}
I20260812 18:59:09.886263 3613 heartbeater.cc:347] Connected to a master server at 127.29.112.254:44267
I20260812 18:59:09.886477 3613 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:09.886897 3613 heartbeater.cc:511] Master 127.29.112.254:44267 requested a full tablet report, sending...
I20260812 18:59:09.887976 3242 ts_manager.cc:194] Registered new tserver with Master: 7ba980c52f7d4e09a06dc2eab7d69a19 (127.29.112.194:41319)
W20260812 18:59:09.988844 3616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:09.989056 3616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:09.989090 3616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:09.992978 3616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:09.993022 3616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:09.993101 3616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:09.997423 3616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:09.998633 3616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:09.999825 3616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:10.009912 3624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:10.009912 3621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:10.009963 3622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:10.010215 3616 server_base.cc:1034] running on GCE node
I20260812 18:59:10.010686 3616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:10.011349 3616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:10.012547 3616 hybrid_clock.cc:648] HybridClock initialized: now 1786561150012524 us; error 43 us; skew 500 ppm
I20260812 18:59:10.014796 3616 webserver.cc:460] Webserver started at http://127.29.112.195:45395/ using document root <none> and password file <none>
I20260812 18:59:10.015376 3616 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:10.015442 3616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:10.015617 3616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:10.017640 3616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "65fc2eae6d7c4b2e92794448be0c00c8"
format_stamp: "Formatted at 2026-08-12 18:59:10 on dist-test-slave-6nmv"
server_key: "ccce4940917c802668fbacc8c2996c0d"
server_key_iv: "608698e53a4793b9504a5d03e94fc757"
server_key_version: "encryptionkey@0"
I20260812 18:59:10.018244 3616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "65fc2eae6d7c4b2e92794448be0c00c8"
format_stamp: "Formatted at 2026-08-12 18:59:10 on dist-test-slave-6nmv"
server_key: "ccce4940917c802668fbacc8c2996c0d"
server_key_iv: "608698e53a4793b9504a5d03e94fc757"
server_key_version: "encryptionkey@0"
I20260812 18:59:10.021953 3616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 18:59:10.024489 3631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:10.025914 3616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 18:59:10.026038 3616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "65fc2eae6d7c4b2e92794448be0c00c8"
format_stamp: "Formatted at 2026-08-12 18:59:10 on dist-test-slave-6nmv"
server_key: "ccce4940917c802668fbacc8c2996c0d"
server_key_iv: "608698e53a4793b9504a5d03e94fc757"
server_key_version: "encryptionkey@0"
I20260812 18:59:10.026147 3616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:10.031284 3616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:10.031909 3616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:10.032099 3616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:10.032648 3616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:10.033553 3616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:10.033608 3616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:10.033674 3616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:10.033707 3616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:10.043174 3616 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:32773
I20260812 18:59:10.043239 3761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:32773 every 8 connection(s)
I20260812 18:59:10.044122 3616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:59:10.049748 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 3616
I20260812 18:59:10.049906 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:10.050251 30147 external_mini_cluster.cc:1428] Setting key e6e4636abb56aa0c42d186e2e8b34627
I20260812 18:59:10.050491 3290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.046003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:56643 (local address 127.29.112.254:44267)
0812 18:59:10.046146 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:10.046151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:10.047179 (+ 1028us) server_negotiation.cc:407] Connection header received
0812 18:59:10.048504 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:10.048507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:10.048555 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:10.048661 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:10.049951 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:10.049956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:10.049963 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:10.049980 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:10.050003 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:10.050014 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:10.050110 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:10.050310 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 18:59:10.050359 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 18:59:10.051592 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.046366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44267 (local address 127.29.112.195:56643)
0812 18:59:10.047024 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:10.047063 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:10.048292 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:10.048819 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:10.048826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:10.048915 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:10.049504 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:10.049507 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:10.049559 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:10.049646 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:10.050149 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:10.050160 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:10.050167 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 18:59:10.050351 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 18:59:10.050636 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":123,"threads_started":1}
I20260812 18:59:10.052762 3762 heartbeater.cc:347] Connected to a master server at 127.29.112.254:44267
I20260812 18:59:10.052991 3762 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:10.053395 3762 heartbeater.cc:511] Master 127.29.112.254:44267 requested a full tablet report, sending...
I20260812 18:59:10.054418 3241 ts_manager.cc:194] Registered new tserver with Master: 65fc2eae6d7c4b2e92794448be0c00c8 (127.29.112.195:32773)
I20260812 18:59:10.064388 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 18:59:10.068240 3290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.066728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44402 (local address 127.29.112.254:44267)
0812 18:59:10.066921 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:10.066924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:10.067024 (+ 100us) server_negotiation.cc:407] Connection header received
0812 18:59:10.067139 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:10.067141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:10.067175 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:10.067239 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:10.067862 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:10.067864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:10.067866 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:10.067876 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:10.067893 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:10.067920 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:10.067988 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:10.068045 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 18:59:10.068076 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 18:59:10.071280 3241 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:59:10.074409 3241 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:59:10.086536 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.082918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:42347 (local address 127.0.0.1:39538)
0812 18:59:10.083589 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:10.083668 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:10.083778 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:10.084994 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:10.084998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:10.085013 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:10.085772 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:10.085774 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:10.085785 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:10.085841 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:10.086155 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:10.086165 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:10.086170 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 18:59:10.086308 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 18:59:10.086379 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":91,"threads_started":1}
I20260812 18:59:10.086536 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.083669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:41319 (local address 127.0.0.1:42370)
0812 18:59:10.084202 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:10.084214 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:10.084295 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:10.084927 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:10.084930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:10.084945 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:10.085580 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:10.085582 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:10.085594 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:10.085627 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:10.086291 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:10.086294 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:10.086297 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 18:59:10.086387 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 18:59:10.086413 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":87,"threads_started":1}
I20260812 18:59:10.086622 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.083175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39538 (local address 127.29.112.193:42347)
0812 18:59:10.084546 (+ 1371us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:10.084563 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:10.084590 (+ 27us) server_negotiation.cc:407] Connection header received
0812 18:59:10.084646 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:10.084653 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:10.084836 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:10.085219 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:10.085982 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:10.085987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:10.086006 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:10.086015 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:10.086047 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:10.086064 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:10.086186 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:10.086395 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 18:59:10.086474 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1258,"thread_start_us":97,"threads_started":1}
I20260812 18:59:10.086627 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.083868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.29.112.194:41319)
0812 18:59:10.084547 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:10.084558 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:10.084580 (+ 22us) server_negotiation.cc:407] Connection header received
0812 18:59:10.084629 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:10.084634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:10.084786 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:10.085192 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:10.086113 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:10.086118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:10.086128 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:10.086138 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:10.086173 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:10.086195 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:10.086303 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:10.086394 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 18:59:10.086468 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":160,"threads_started":1}
I20260812 18:59:10.087265 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.083577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37860 (local address 127.29.112.195:32773)
0812 18:59:10.084061 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:10.084073 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:10.084296 (+ 223us) server_negotiation.cc:407] Connection header received
0812 18:59:10.084371 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:10.084378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:10.084633 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:10.084768 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:10.086113 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:10.086118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:10.086128 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:10.086138 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:10.086173 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:10.086195 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:10.086303 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:10.087027 (+ 724us) server_negotiation.cc:299] Negotiation successful
0812 18:59:10.087104 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":181,"threads_started":1}
I20260812 18:59:10.087651 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.083160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:32773 (local address 127.0.0.1:37860)
0812 18:59:10.084173 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:10.084187 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:10.084279 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:10.084766 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:10.084772 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:10.084843 (+ 71us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:10.085424 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:10.085426 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:10.085447 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:10.085529 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:10.086579 (+ 1050us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:10.086584 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:10.086886 (+ 302us) client_negotiation.cc:769] Sending connection context
0812 18:59:10.087479 (+ 593us) client_negotiation.cc:240] Negotiation successful
0812 18:59:10.087519 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"thread_start_us":110,"threads_started":1}
I20260812 18:59:10.089638 3386 tablet_service.cc:1467] Processing CreateTablet for tablet c6545b394a8c4aa699f3c06b03d1e900 (DEFAULT_TABLE table=test-table [id=d0c68b85d40e48e5ab0b1a2d606556fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:10.089638 3536 tablet_service.cc:1467] Processing CreateTablet for tablet c6545b394a8c4aa699f3c06b03d1e900 (DEFAULT_TABLE table=test-table [id=d0c68b85d40e48e5ab0b1a2d606556fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:10.089779 3683 tablet_service.cc:1467] Processing CreateTablet for tablet c6545b394a8c4aa699f3c06b03d1e900 (DEFAULT_TABLE table=test-table [id=d0c68b85d40e48e5ab0b1a2d606556fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:10.090804 3386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6545b394a8c4aa699f3c06b03d1e900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:10.090804 3536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6545b394a8c4aa699f3c06b03d1e900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:10.090806 3683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6545b394a8c4aa699f3c06b03d1e900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:10.096843 3783 tablet_bootstrap.cc:492] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: Bootstrap starting.
I20260812 18:59:10.098448 3785 tablet_bootstrap.cc:492] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9: Bootstrap starting.
I20260812 18:59:10.099531 3783 tablet_bootstrap.cc:654] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:10.100240 3783 log.cc:824] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:10.100396 3785 tablet_bootstrap.cc:654] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:10.101156 3785 log.cc:824] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:10.101658 3784 tablet_bootstrap.cc:492] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: Bootstrap starting.
I20260812 18:59:10.102392 3783 tablet_bootstrap.cc:492] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: No bootstrap required, opened a new log
I20260812 18:59:10.102602 3783 ts_tablet_manager.cc:1397] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 18:59:10.102881 3785 tablet_bootstrap.cc:492] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9: No bootstrap required, opened a new log
I20260812 18:59:10.103078 3785 ts_tablet_manager.cc:1397] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 18:59:10.103742 3784 tablet_bootstrap.cc:654] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:10.104514 3784 log.cc:824] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:10.106508 3784 tablet_bootstrap.cc:492] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: No bootstrap required, opened a new log
I20260812 18:59:10.106676 3784 ts_tablet_manager.cc:1397] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 18:59:10.107101 3783 raft_consensus.cc:348] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6755591b5648cbb6b15fda540262d9" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 42347 } } peers { permanent_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 32773 } } peers { permanent_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 41319 } }
I20260812 18:59:10.107326 3783 raft_consensus.cc:374] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:10.107187 3785 raft_consensus.cc:348] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6755591b5648cbb6b15fda540262d9" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 42347 } } peers { permanent_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 32773 } } peers { permanent_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 41319 } }
I20260812 18:59:10.107378 3783 raft_consensus.cc:729] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65fc2eae6d7c4b2e92794448be0c00c8, State: Initialized, Role: FOLLOWER
I20260812 18:59:10.107419 3785 raft_consensus.cc:374] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:10.107481 3785 raft_consensus.cc:729] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6755591b5648cbb6b15fda540262d9, State: Initialized, Role: FOLLOWER
I20260812 18:59:10.107800 3783 consensus_queue.cc:260] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6755591b5648cbb6b15fda540262d9" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 42347 } } peers { permanent_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 32773 } } peers { permanent_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 41319 } }
I20260812 18:59:10.107906 3785 consensus_queue.cc:260] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6755591b5648cbb6b15fda540262d9" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 42347 } } peers { permanent_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 32773 } } peers { permanent_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 41319 } }
I20260812 18:59:10.108436 3762 heartbeater.cc:503] Master 127.29.112.254:44267 was elected leader, sending a full tablet report...
I20260812 18:59:10.108745 3783 ts_tablet_manager.cc:1428] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 18:59:10.110001 3785 ts_tablet_manager.cc:1428] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 18:59:10.110256 3460 heartbeater.cc:503] Master 127.29.112.254:44267 was elected leader, sending a full tablet report...
I20260812 18:59:10.110914 3784 raft_consensus.cc:348] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6755591b5648cbb6b15fda540262d9" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 42347 } } peers { permanent_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 32773 } } peers { permanent_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 41319 } }
I20260812 18:59:10.111161 3784 raft_consensus.cc:374] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:10.111470 3784 raft_consensus.cc:729] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba980c52f7d4e09a06dc2eab7d69a19, State: Initialized, Role: FOLLOWER
I20260812 18:59:10.111882 3784 consensus_queue.cc:260] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6755591b5648cbb6b15fda540262d9" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 42347 } } peers { permanent_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 32773 } } peers { permanent_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 41319 } }
I20260812 18:59:10.112859 3784 ts_tablet_manager.cc:1428] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260812 18:59:10.112877 3613 heartbeater.cc:503] Master 127.29.112.254:44267 was elected leader, sending a full tablet report...
W20260812 18:59:10.128544 3614 tablet.cc:2369] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 18:59:10.219565 3463 tablet.cc:2369] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 18:59:10.295204 3763 tablet.cc:2369] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:59:10.324985 3792 raft_consensus.cc:482] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:59:10.325289 3792 raft_consensus.cc:504] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6755591b5648cbb6b15fda540262d9" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 42347 } } peers { permanent_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 32773 } } peers { permanent_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 41319 } }
I20260812 18:59:10.326577 3792 leader_election.cc:288] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff6755591b5648cbb6b15fda540262d9 (127.29.112.193:42347), 65fc2eae6d7c4b2e92794448be0c00c8 (127.29.112.195:32773)
I20260812 18:59:10.329505 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.326985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:42347 (local address 127.29.112.194:54857)
0812 18:59:10.327117 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:10.327137 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:10.327232 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:10.327617 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:10.327621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:10.327639 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:10.328261 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:10.328263 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:10.328276 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:10.328304 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:10.328980 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:10.328988 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:10.328995 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 18:59:10.329224 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 18:59:10.329295 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 18:59:10.329548 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.327108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:54857 (local address 127.29.112.193:42347)
0812 18:59:10.327257 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:10.327264 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:10.327285 (+ 21us) server_negotiation.cc:407] Connection header received
0812 18:59:10.327337 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:10.327342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:10.327380 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:10.327485 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:10.328616 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:10.328622 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:10.328627 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:10.328644 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:10.328667 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:10.328679 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:10.328805 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:10.329294 (+ 489us) server_negotiation.cc:299] Negotiation successful
0812 18:59:10.329350 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 18:59:10.329860 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.327244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:44377 (local address 127.29.112.195:32773)
0812 18:59:10.327383 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:10.327386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:10.327903 (+ 517us) server_negotiation.cc:407] Connection header received
0812 18:59:10.327985 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:10.327990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:10.328047 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:10.328135 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:10.329303 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:10.329306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:10.329309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:10.329324 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:10.329349 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:10.329359 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:10.329480 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:10.329630 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 18:59:10.329689 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 18:59:10.329881 3793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.326985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:32773 (local address 127.29.112.194:44377)
0812 18:59:10.327774 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:10.327791 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:10.327901 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:10.328242 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:10.328246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:10.328262 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:10.329028 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:10.329030 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:10.329052 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:10.329102 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:10.329487 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:10.329491 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:10.329494 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 18:59:10.329604 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 18:59:10.329645 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":304,"thread_start_us":172,"threads_started":1}
I20260812 18:59:10.330562 3411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6545b394a8c4aa699f3c06b03d1e900" candidate_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6755591b5648cbb6b15fda540262d9" is_pre_election: true
I20260812 18:59:10.330874 3705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6545b394a8c4aa699f3c06b03d1e900" candidate_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" is_pre_election: true
I20260812 18:59:10.330937 3411 raft_consensus.cc:2454] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba980c52f7d4e09a06dc2eab7d69a19 in term 0.
I20260812 18:59:10.331220 3705 raft_consensus.cc:2454] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba980c52f7d4e09a06dc2eab7d69a19 in term 0.
I20260812 18:59:10.331568 3481 leader_election.cc:302] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba980c52f7d4e09a06dc2eab7d69a19, ff6755591b5648cbb6b15fda540262d9; no voters:
I20260812 18:59:10.331948 3792 raft_consensus.cc:2781] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:59:10.332046 3792 raft_consensus.cc:482] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:59:10.332080 3792 raft_consensus.cc:3037] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:10.333259 3792 raft_consensus.cc:504] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6755591b5648cbb6b15fda540262d9" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 42347 } } peers { permanent_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 32773 } } peers { permanent_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 41319 } }
I20260812 18:59:10.333719 3792 leader_election.cc:288] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [CANDIDATE]: Term 1 election: Requested vote from peers ff6755591b5648cbb6b15fda540262d9 (127.29.112.193:42347), 65fc2eae6d7c4b2e92794448be0c00c8 (127.29.112.195:32773)
I20260812 18:59:10.334291 3411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6545b394a8c4aa699f3c06b03d1e900" candidate_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6755591b5648cbb6b15fda540262d9"
I20260812 18:59:10.334571 3705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6545b394a8c4aa699f3c06b03d1e900" candidate_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65fc2eae6d7c4b2e92794448be0c00c8"
I20260812 18:59:10.334563 3411 raft_consensus.cc:3037] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:10.334728 3705 raft_consensus.cc:3037] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:10.336295 3705 raft_consensus.cc:2454] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba980c52f7d4e09a06dc2eab7d69a19 in term 1.
I20260812 18:59:10.336299 3411 raft_consensus.cc:2454] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba980c52f7d4e09a06dc2eab7d69a19 in term 1.
I20260812 18:59:10.336694 3482 leader_election.cc:302] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65fc2eae6d7c4b2e92794448be0c00c8, 7ba980c52f7d4e09a06dc2eab7d69a19; no voters:
I20260812 18:59:10.336973 3792 raft_consensus.cc:2781] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:10.337337 3792 raft_consensus.cc:686] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 1 LEADER]: Becoming Leader. State: Replica: 7ba980c52f7d4e09a06dc2eab7d69a19, State: Running, Role: LEADER
I20260812 18:59:10.337733 3792 consensus_queue.cc:237] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6755591b5648cbb6b15fda540262d9" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 42347 } } peers { permanent_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 32773 } } peers { permanent_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 41319 } }
I20260812 18:59:10.341337 3242 catalog_manager.cc:5495] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ba980c52f7d4e09a06dc2eab7d69a19 (127.29.112.194). New cstate: current_term: 1 leader_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6755591b5648cbb6b15fda540262d9" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 42347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ba980c52f7d4e09a06dc2eab7d69a19" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 41319 } health_report { overall_health: HEALTHY } } }
I20260812 18:59:10.377910 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.376260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42380 (local address 127.29.112.194:41319)
0812 18:59:10.376415 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:10.376420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:10.376435 (+ 15us) server_negotiation.cc:407] Connection header received
0812 18:59:10.376616 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:10.376619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:10.376691 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:10.376788 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:10.377452 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:10.377455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:10.377458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:10.377469 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:10.377495 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:10.377509 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:10.377600 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:10.377698 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 18:59:10.377737 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 18:59:10.383739 3411 raft_consensus.cc:1261] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9 [term 1 FOLLOWER]: Refusing update from remote peer 7ba980c52f7d4e09a06dc2eab7d69a19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:10.383757 3705 raft_consensus.cc:1261] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8 [term 1 FOLLOWER]: Refusing update from remote peer 7ba980c52f7d4e09a06dc2eab7d69a19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:10.384567 3794 consensus_queue.cc:1035] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff6755591b5648cbb6b15fda540262d9" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 42347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:10.384752 3792 consensus_queue.cc:1035] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65fc2eae6d7c4b2e92794448be0c00c8" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:10.392771 3800 mvcc.cc:205] Tried to move back new op lower bound from 7317754471964446720 to 7317754471786446848. Current Snapshot: MvccSnapshot[applied={T|T < 7317754471964446720}]
I20260812 18:59:10.400161 3241 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:10.400290 3241 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:10.402626 3241 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ec64504252f4c0590b677c162f2e40a: Sending DeleteTablet for 3 replicas of tablet c6545b394a8c4aa699f3c06b03d1e900
I20260812 18:59:10.403435 3386 tablet_service.cc:1514] Processing DeleteTablet for tablet c6545b394a8c4aa699f3c06b03d1e900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:10 UTC) from {username='slave'} at 127.0.0.1:39538
I20260812 18:59:10.403528 3683 tablet_service.cc:1514] Processing DeleteTablet for tablet c6545b394a8c4aa699f3c06b03d1e900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:10 UTC) from {username='slave'} at 127.0.0.1:37860
I20260812 18:59:10.403931 3808 tablet_replica.cc:330] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9: stopping tablet replica
I20260812 18:59:10.403910 3536 tablet_service.cc:1514] Processing DeleteTablet for tablet c6545b394a8c4aa699f3c06b03d1e900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:10 UTC) from {username='slave'} at 127.0.0.1:42370
I20260812 18:59:10.404318 3809 tablet_replica.cc:330] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: stopping tablet replica
I20260812 18:59:10.404330 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 3314
I20260812 18:59:10.404318 3808 raft_consensus.cc:2229] T c6545b394a8c4aa699f3c06b03d1e900 P ff6755591b5648cbb6b15fda540262d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:10.404608 3809 raft_consensus.cc:2229] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:10.404878 3809 raft_consensus.cc:2258] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:10.405963 3810 tablet_replica.cc:330] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: stopping tablet replica
I20260812 18:59:10.406301 3810 raft_consensus.cc:2229] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:59:10.406705 3810 raft_consensus.cc:2258] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:10.407140 3809 ts_tablet_manager.cc:1905] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:10.408061 3810 ts_tablet_manager.cc:1905] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:10.409716 3809 ts_tablet_manager.cc:1918] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:10.409822 3809 log.cc:1196] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/c6545b394a8c4aa699f3c06b03d1e900
I20260812 18:59:10.410127 3809 ts_tablet_manager.cc:1939] T c6545b394a8c4aa699f3c06b03d1e900 P 65fc2eae6d7c4b2e92794448be0c00c8: Deleting consensus metadata
I20260812 18:59:10.411180 3224 catalog_manager.cc:4841] TS 65fc2eae6d7c4b2e92794448be0c00c8 (127.29.112.195:32773): tablet c6545b394a8c4aa699f3c06b03d1e900 (table test-table [id=d0c68b85d40e48e5ab0b1a2d606556fc]) successfully deleted
I20260812 18:59:10.411640 3810 ts_tablet_manager.cc:1918] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:10.411741 3810 log.cc:1196] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/c6545b394a8c4aa699f3c06b03d1e900
I20260812 18:59:10.412061 3810 ts_tablet_manager.cc:1939] T c6545b394a8c4aa699f3c06b03d1e900 P 7ba980c52f7d4e09a06dc2eab7d69a19: Deleting consensus metadata
I20260812 18:59:10.413307 3224 catalog_manager.cc:4841] TS 7ba980c52f7d4e09a06dc2eab7d69a19 (127.29.112.194:41319): tablet c6545b394a8c4aa699f3c06b03d1e900 (table test-table [id=d0c68b85d40e48e5ab0b1a2d606556fc]) successfully deleted
W20260812 18:59:10.416854 3223 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:42347 (error 108)
I20260812 18:59:10.417677 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 3465
I20260812 18:59:10.417917 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.417583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:42347 (local address 127.0.0.1:39544)
0812 18:59:10.417708 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:10.417802 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:42347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 18:59:10.422981 3223 catalog_manager.cc:4584] TS ff6755591b5648cbb6b15fda540262d9 (127.29.112.193:42347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6545b394a8c4aa699f3c06b03d1e900: Network error: Client connection negotiation failed: client connection to 127.29.112.193:42347: connect: Connection refused (error 111)
I20260812 18:59:10.427950 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 3616
I20260812 18:59:10.436664 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 3207
2026-08-12T18:59:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1123 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 18:59:10.460295 30147 mini_oidc.cc:152] Starting JWKS server
I20260812 18:59:10.461978 30147 webserver.cc:460] Webserver started at https://127.0.0.1:35985/ using document root <none> and password file <none>
I20260812 18:59:10.462162 30147 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 18:59:10.462442 30147 webserver.cc:460] Webserver started at http://127.0.0.1:38477/ using document root <none> and password file <none>
2026-08-12T18:59:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:10Z Disabled control of system clock
I20260812 18:59:10.471873 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:41325
--webserver_interface=127.29.112.254
--webserver_port=0
--builtin_ntp_servers=127.29.112.212:41447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:41325
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35985/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/testchainca.cert with env {}
W20260812 18:59:10.579823 3817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:10.580049 3817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:10.580106 3817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:10.583817 3817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:10.583858 3817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:10.583873 3817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:10.583891 3817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:10.583906 3817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 18:59:10.583920 3817 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 18:59:10.583933 3817 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35985/jwks.json
I20260812 18:59:10.588964 3817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:41447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:41325
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:41325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35985/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:10.589783 3817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:10.590701 3817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:10.599030 3828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:10.599227 3824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:10.599032 3825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:10.599205 3817 server_base.cc:1034] running on GCE node
I20260812 18:59:10.599875 3817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:10.600868 3817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:10.602057 3817 hybrid_clock.cc:648] HybridClock initialized: now 1786561150602044 us; error 39 us; skew 500 ppm
I20260812 18:59:10.603986 3817 webserver.cc:460] Webserver started at http://127.29.112.254:43471/ using document root <none> and password file <none>
I20260812 18:59:10.604553 3817 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:10.604646 3817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:10.604868 3817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:10.607287 3817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "2eb1023f75024097a5758c978fda7c23"
format_stamp: "Formatted at 2026-08-12 18:59:10 on dist-test-slave-6nmv"
server_key: "7f4e987893e96ab2e4b8d9cf5523004a"
server_key_iv: "13e2bb17024a036e3b7c5407e478a1bf"
server_key_version: "encryptionkey@0"
I20260812 18:59:10.607997 3817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "2eb1023f75024097a5758c978fda7c23"
format_stamp: "Formatted at 2026-08-12 18:59:10 on dist-test-slave-6nmv"
server_key: "7f4e987893e96ab2e4b8d9cf5523004a"
server_key_iv: "13e2bb17024a036e3b7c5407e478a1bf"
server_key_version: "encryptionkey@0"
I20260812 18:59:10.612457 3817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 18:59:10.614766 3835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:10.615885 3817 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:10.616029 3817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "2eb1023f75024097a5758c978fda7c23"
format_stamp: "Formatted at 2026-08-12 18:59:10 on dist-test-slave-6nmv"
server_key: "7f4e987893e96ab2e4b8d9cf5523004a"
server_key_iv: "13e2bb17024a036e3b7c5407e478a1bf"
server_key_version: "encryptionkey@0"
I20260812 18:59:10.616137 3817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:10.636619 3817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:10.637285 3817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:10.637481 3817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:10.644230 3817 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:41325
I20260812 18:59:10.644239 3896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:41325 every 8 connection(s)
I20260812 18:59:10.645151 3817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:10.647324 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 3817
I20260812 18:59:10.647457 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:10.647748 30147 external_mini_cluster.cc:1428] Setting key 5564b252b9c34098ce92f3e57f092a60
I20260812 18:59:10.648484 3897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:10.654069 3897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23: Bootstrap starting.
I20260812 18:59:10.656366 3897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:10.656491 3900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.648959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.29.112.254:41325)
0812 18:59:10.649613 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:10.649627 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:10.649659 (+ 32us) server_negotiation.cc:407] Connection header received
0812 18:59:10.650353 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:10.650367 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:10.650753 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:10.651079 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:10.651486 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:10.652424 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:10.653324 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:10.653750 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:10.653929 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:10.654828 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:10.654833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:10.654844 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:10.655203 (+ 359us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:10.655252 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:10.655271 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:10.655379 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:10.655562 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 18:59:10.655803 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":134,"threads_started":1}
I20260812 18:59:10.657166 3897 log.cc:824] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:10.659169 3897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23: No bootstrap required, opened a new log
I20260812 18:59:10.659276 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 18:59:10.662891 3897 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb1023f75024097a5758c978fda7c23" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41325 } }
I20260812 18:59:10.663120 3897 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:10.663192 3897 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eb1023f75024097a5758c978fda7c23, State: Initialized, Role: FOLLOWER
I20260812 18:59:10.663578 3897 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb1023f75024097a5758c978fda7c23" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41325 } }
I20260812 18:59:10.663715 3897 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:10.663785 3897 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:10.663888 3897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:10.664702 3897 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb1023f75024097a5758c978fda7c23" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41325 } }
I20260812 18:59:10.665028 3897 leader_election.cc:302] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eb1023f75024097a5758c978fda7c23; no voters:
I20260812 18:59:10.665308 3897 leader_election.cc:288] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:10.665469 3902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:10.665696 3902 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [term 1 LEADER]: Becoming Leader. State: Replica: 2eb1023f75024097a5758c978fda7c23, State: Running, Role: LEADER
I20260812 18:59:10.666007 3902 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb1023f75024097a5758c978fda7c23" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41325 } }
I20260812 18:59:10.666529 3897 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:10.667578 3904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eb1023f75024097a5758c978fda7c23. Latest consensus state: current_term: 1 leader_uuid: "2eb1023f75024097a5758c978fda7c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb1023f75024097a5758c978fda7c23" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41325 } } }
I20260812 18:59:10.667702 3904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:10.667997 3903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2eb1023f75024097a5758c978fda7c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb1023f75024097a5758c978fda7c23" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41325 } } }
I20260812 18:59:10.668104 3915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:10.669040 3903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:10.671238 3915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:10.676640 3915 catalog_manager.cc:1347] Generated new cluster ID: 12802789a4524164bfcef3234701c30e
I20260812 18:59:10.676720 3915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:10.689205 3915 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:10.690291 3915 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:10.703630 3915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2eb1023f75024097a5758c978fda7c23: Generated new TSK 0
I20260812 18:59:10.704334 3915 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 18:59:10.738056 3935 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 18:59:10.738179 3935 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 18:59:10.738734 3900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.716848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43358 (local address 127.29.112.254:41325)
0812 18:59:10.716964 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:10.716971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:10.717181 (+ 210us) server_negotiation.cc:407] Connection header received
0812 18:59:10.717379 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:10.717381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:10.717422 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:10.717504 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0812 18:59:10.717778 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:10.718296 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:10.719375 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:10.719610 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:10.719700 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:10.720005 (+ 305us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 18:59:10.738326 (+ 18321us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35985/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 18:59:10.738599 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43358: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35985/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 18:59:10.738797 3900 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43358: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35985/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 18:59:10.739992 3934 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:10.716706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:41325 (local address 127.0.0.1:43358)
0812 18:59:10.717084 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:10.717097 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:10.717235 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:10.717539 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:10.717542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:10.717549 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0812 18:59:10.717682 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:10.717688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:10.718393 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:10.718396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:10.719276 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:10.719285 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:10.719714 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:10.719718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:10.719809 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:10.719855 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 18:59:10.738535 (+ 18680us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35985/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 18:59:10.738725 (+ 190us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.112.254:41325: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35985/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":122,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.112.254:41325: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35985/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 18:59:10.740753 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 3817
2026-08-12T18:59:10Z chronyd exiting
I20260812 18:59:10.781498 30147 test_util.cc:175] -----------------------------------------------
I20260812 18:59:10.781580 30147 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786561125017040-30147-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (322 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 18:59:10.782691 30147 mini_oidc.cc:152] Starting JWKS server
I20260812 18:59:10.783504 30147 webserver.cc:460] Webserver started at https://127.0.0.1:41351/ using document root <none> and password file <none>
I20260812 18:59:10.783581 30147 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 18:59:10.783782 30147 webserver.cc:460] Webserver started at http://127.0.0.1:37487/ using document root <none> and password file <none>
2026-08-12T18:59:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:10Z Disabled control of system clock
I20260812 18:59:10.793617 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:34335
--webserver_interface=127.29.112.254
--webserver_port=0
--builtin_ntp_servers=127.29.112.212:42951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:34335
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41351/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/testchainca.cert with env {}
W20260812 18:59:10.896842 3941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:10.897054 3941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:10.897087 3941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:10.900609 3941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:10.900656 3941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:10.900671 3941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:10.900684 3941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:10.900697 3941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 18:59:10.900712 3941 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 18:59:10.900723 3941 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41351/jwks.json
I20260812 18:59:10.906023 3941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:42951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:34335
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:34335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41351/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:10.906944 3941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:10.907845 3941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:10.916440 3947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:10.916527 3941 server_base.cc:1034] running on GCE node
W20260812 18:59:10.916397 3952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:10.916601 3948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:10.917320 3941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:10.918610 3941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:10.919796 3941 hybrid_clock.cc:648] HybridClock initialized: now 1786561150919777 us; error 33 us; skew 500 ppm
I20260812 18:59:10.921558 3941 webserver.cc:460] Webserver started at http://127.29.112.254:37279/ using document root <none> and password file <none>
I20260812 18:59:10.922164 3941 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:10.922250 3941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:10.922442 3941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:10.924474 3941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "d9ee3c1f83834d1dad9ac302581fcf97"
format_stamp: "Formatted at 2026-08-12 18:59:10 on dist-test-slave-6nmv"
server_key: "6b9fd6afeb69db57081c09e504e2188a"
server_key_iv: "aaa1769895853e451be963dcfdd3a9b2"
server_key_version: "encryptionkey@0"
I20260812 18:59:10.925046 3941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "d9ee3c1f83834d1dad9ac302581fcf97"
format_stamp: "Formatted at 2026-08-12 18:59:10 on dist-test-slave-6nmv"
server_key: "6b9fd6afeb69db57081c09e504e2188a"
server_key_iv: "aaa1769895853e451be963dcfdd3a9b2"
server_key_version: "encryptionkey@0"
I20260812 18:59:10.928786 3941 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 18:59:10.931092 3960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:10.932291 3941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:10.932443 3941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "d9ee3c1f83834d1dad9ac302581fcf97"
format_stamp: "Formatted at 2026-08-12 18:59:10 on dist-test-slave-6nmv"
server_key: "6b9fd6afeb69db57081c09e504e2188a"
server_key_iv: "aaa1769895853e451be963dcfdd3a9b2"
server_key_version: "encryptionkey@0"
I20260812 18:59:10.932554 3941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:10.957612 3941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:10.958287 3941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:10.958491 3941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:10.965185 3941 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:34335
I20260812 18:59:10.965195 4017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:34335 every 8 connection(s)
I20260812 18:59:10.966118 3941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:10.969091 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 3941
I20260812 18:59:10.969203 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:10.969225 4018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:10.969493 30147 external_mini_cluster.cc:1428] Setting key 41b5fc85c143f17d223623cf2ec832a0
I20260812 18:59:10.975656 4018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97: Bootstrap starting.
I20260812 18:59:10.978143 4018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:10.978677 4021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:10.970714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42842 (local address 127.29.112.254:34335)
0812 18:59:10.971252 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:10.971265 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:10.971297 (+ 32us) server_negotiation.cc:407] Connection header received
0812 18:59:10.972153 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:10.972170 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:10.972545 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:10.972814 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:10.973121 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:10.974172 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:10.975431 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:10.976043 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:10.976189 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:10.977068 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:10.977072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:10.977084 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:10.977430 (+ 346us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:10.977480 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:10.977498 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:10.977572 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:10.977708 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 18:59:10.978018 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":237,"thread_start_us":134,"threads_started":1}
I20260812 18:59:10.978832 4018 log.cc:824] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:10.980616 4018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97: No bootstrap required, opened a new log
I20260812 18:59:10.981197 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 18:59:10.984318 4018 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ee3c1f83834d1dad9ac302581fcf97" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 34335 } }
I20260812 18:59:10.984516 4018 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:10.984563 4018 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9ee3c1f83834d1dad9ac302581fcf97, State: Initialized, Role: FOLLOWER
I20260812 18:59:10.984938 4018 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ee3c1f83834d1dad9ac302581fcf97" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 34335 } }
I20260812 18:59:10.985056 4018 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:10.985095 4018 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:10.985162 4018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:10.986032 4018 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ee3c1f83834d1dad9ac302581fcf97" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 34335 } }
I20260812 18:59:10.986304 4018 leader_election.cc:302] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9ee3c1f83834d1dad9ac302581fcf97; no voters:
I20260812 18:59:10.986547 4018 leader_election.cc:288] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:10.986749 4023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:10.987084 4023 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [term 1 LEADER]: Becoming Leader. State: Replica: d9ee3c1f83834d1dad9ac302581fcf97, State: Running, Role: LEADER
I20260812 18:59:10.987421 4023 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ee3c1f83834d1dad9ac302581fcf97" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 34335 } }
I20260812 18:59:10.987736 4018 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:10.989074 4025 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9ee3c1f83834d1dad9ac302581fcf97. Latest consensus state: current_term: 1 leader_uuid: "d9ee3c1f83834d1dad9ac302581fcf97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ee3c1f83834d1dad9ac302581fcf97" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 34335 } } }
I20260812 18:59:10.989207 4025 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:10.989739 4034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:10.990207 4024 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9ee3c1f83834d1dad9ac302581fcf97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ee3c1f83834d1dad9ac302581fcf97" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 34335 } } }
I20260812 18:59:10.990317 4024 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:10.992729 4034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:10.998021 4034 catalog_manager.cc:1347] Generated new cluster ID: 5f710e6d9da343a3a062fdb4c337ff9c
I20260812 18:59:10.998091 4034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:11.004464 4034 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:11.005249 4034 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:11.012035 4034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9ee3c1f83834d1dad9ac302581fcf97: Generated new TSK 0
I20260812 18:59:11.012775 4034 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 18:59:11.032783 4049 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 18:59:11.032858 4049 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 18:59:11.033502 4021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:11.019140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42858 (local address 127.29.112.254:34335)
0812 18:59:11.019265 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:11.019270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:11.019421 (+ 151us) server_negotiation.cc:407] Connection header received
0812 18:59:11.019600 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:11.019602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:11.019643 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:11.019738 (+ 95us) server_negotiation.cc:226] Negotiated authn=JWT
0812 18:59:11.019988 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.020437 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.021360 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.021600 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.021713 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:11.022058 (+ 345us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 18:59:11.033113 (+ 11055us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 18:59:11.033354 (+ 241us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42858: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 18:59:11.033566 4021 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42858: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 18:59:11.033584 4048 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:11.019072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:34335 (local address 127.0.0.1:42858)
0812 18:59:11.019346 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:11.019357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:11.019465 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:11.019738 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:11.019741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:11.019748 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0812 18:59:11.019869 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:11.019877 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.020559 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.020563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:11.021228 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:11.021235 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.021726 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.021730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:11.021853 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:11.021879 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 18:59:11.033267 (+ 11388us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 18:59:11.033414 (+ 147us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.112.254:34335: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":108,"threads_started":1}
I20260812 18:59:11.034282 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 3941
2026-08-12T18:59:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (295 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 18:59:11.078343 30147 mini_oidc.cc:152] Starting JWKS server
I20260812 18:59:11.079203 30147 webserver.cc:460] Webserver started at https://127.0.0.1:35065/ using document root <none> and password file <none>
I20260812 18:59:11.079334 30147 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 18:59:11.079604 30147 webserver.cc:460] Webserver started at http://127.0.0.1:37711/ using document root <none> and password file <none>
2026-08-12T18:59:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:11Z Disabled control of system clock
I20260812 18:59:11.089354 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:41479
--webserver_interface=127.29.112.254
--webserver_port=0
--builtin_ntp_servers=127.29.112.212:33933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:41479
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35065/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 18:59:11.194701 4055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:11.194933 4055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:11.194967 4055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:11.198621 4055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:11.198660 4055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:11.198673 4055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:11.198688 4055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:11.198699 4055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 18:59:11.198712 4055 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 18:59:11.198724 4055 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35065/jwks.json
I20260812 18:59:11.202803 4055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:33933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:41479
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:41479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35065/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:11.203881 4055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:11.205088 4055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:11.213302 4063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:11.213310 4061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:11.213460 4055 server_base.cc:1034] running on GCE node
W20260812 18:59:11.213606 4066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:11.214237 4055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:11.215327 4055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:11.216485 4055 hybrid_clock.cc:648] HybridClock initialized: now 1786561151216457 us; error 42 us; skew 500 ppm
I20260812 18:59:11.218302 4055 webserver.cc:460] Webserver started at http://127.29.112.254:42531/ using document root <none> and password file <none>
I20260812 18:59:11.218842 4055 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:11.218902 4055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:11.219103 4055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:11.221096 4055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "d77ce00c5ed24e1c8405de6a2f5486ad"
format_stamp: "Formatted at 2026-08-12 18:59:11 on dist-test-slave-6nmv"
server_key: "fb46dc1dfa3166e86405be6f27892e35"
server_key_iv: "9bfc60a304173ac88bfbd9591ee6c6f8"
server_key_version: "encryptionkey@0"
I20260812 18:59:11.221676 4055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "d77ce00c5ed24e1c8405de6a2f5486ad"
format_stamp: "Formatted at 2026-08-12 18:59:11 on dist-test-slave-6nmv"
server_key: "fb46dc1dfa3166e86405be6f27892e35"
server_key_iv: "9bfc60a304173ac88bfbd9591ee6c6f8"
server_key_version: "encryptionkey@0"
I20260812 18:59:11.225323 4055 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 18:59:11.227790 4072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:11.229084 4055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 18:59:11.229206 4055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "d77ce00c5ed24e1c8405de6a2f5486ad"
format_stamp: "Formatted at 2026-08-12 18:59:11 on dist-test-slave-6nmv"
server_key: "fb46dc1dfa3166e86405be6f27892e35"
server_key_iv: "9bfc60a304173ac88bfbd9591ee6c6f8"
server_key_version: "encryptionkey@0"
I20260812 18:59:11.229305 4055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:11.247278 4055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:11.247921 4055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:11.248116 4055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:11.255002 4055 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:41479
I20260812 18:59:11.255062 4136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:41479 every 8 connection(s)
I20260812 18:59:11.255955 4055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:11.259375 4137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:11.264902 4137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad: Bootstrap starting.
I20260812 18:59:11.265050 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 4055
I20260812 18:59:11.265192 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:11.265462 30147 external_mini_cluster.cc:1428] Setting key d16cf637d01b4cc24e2f94450da3041f
I20260812 18:59:11.267877 4137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:11.268662 4137 log.cc:824] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:11.270658 4137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad: No bootstrap required, opened a new log
I20260812 18:59:11.274405 4140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:11.266844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50318 (local address 127.29.112.254:41479)
0812 18:59:11.267472 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:11.267486 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:11.267519 (+ 33us) server_negotiation.cc:407] Connection header received
0812 18:59:11.268160 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:11.268177 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:11.268548 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:11.268861 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:11.269284 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.270200 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.271165 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.271606 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.271825 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:11.272740 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:11.272745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:11.272757 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:11.273133 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:11.273185 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:11.273204 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:11.273311 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:11.273457 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 18:59:11.273691 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":137,"threads_started":1}
I20260812 18:59:11.274811 4137 raft_consensus.cc:348] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77ce00c5ed24e1c8405de6a2f5486ad" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41479 } }
I20260812 18:59:11.275095 4137 raft_consensus.cc:374] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:11.275157 4137 raft_consensus.cc:729] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77ce00c5ed24e1c8405de6a2f5486ad, State: Initialized, Role: FOLLOWER
I20260812 18:59:11.275628 4137 consensus_queue.cc:260] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77ce00c5ed24e1c8405de6a2f5486ad" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41479 } }
I20260812 18:59:11.275765 4137 raft_consensus.cc:388] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:11.275861 4137 raft_consensus.cc:482] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:11.276005 4137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:11.277205 4137 raft_consensus.cc:504] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77ce00c5ed24e1c8405de6a2f5486ad" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41479 } }
I20260812 18:59:11.277413 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 18:59:11.277616 4137 leader_election.cc:302] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d77ce00c5ed24e1c8405de6a2f5486ad; no voters:
I20260812 18:59:11.277979 4137 leader_election.cc:288] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:11.278036 4142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:11.278226 4142 raft_consensus.cc:686] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [term 1 LEADER]: Becoming Leader. State: Replica: d77ce00c5ed24e1c8405de6a2f5486ad, State: Running, Role: LEADER
I20260812 18:59:11.278546 4142 consensus_queue.cc:237] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77ce00c5ed24e1c8405de6a2f5486ad" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41479 } }
I20260812 18:59:11.279247 4137 sys_catalog.cc:564] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:11.280288 4143 sys_catalog.cc:455] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d77ce00c5ed24e1c8405de6a2f5486ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77ce00c5ed24e1c8405de6a2f5486ad" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41479 } } }
I20260812 18:59:11.280416 4143 sys_catalog.cc:458] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:11.280799 4152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:11.280769 4144 sys_catalog.cc:455] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader d77ce00c5ed24e1c8405de6a2f5486ad. Latest consensus state: current_term: 1 leader_uuid: "d77ce00c5ed24e1c8405de6a2f5486ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d77ce00c5ed24e1c8405de6a2f5486ad" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41479 } } }
I20260812 18:59:11.281531 4144 sys_catalog.cc:458] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:11.284130 4152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:11.289593 4152 catalog_manager.cc:1347] Generated new cluster ID: d59647975e6c4228a0c6e9b5b78b14bf
I20260812 18:59:11.289667 4152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:11.307207 4152 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:11.308315 4152 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:11.320719 4152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d77ce00c5ed24e1c8405de6a2f5486ad: Generated new TSK 0
I20260812 18:59:11.321472 4152 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 18:59:11.345280 4140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:11.339799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50334 (local address 127.29.112.254:41479)
0812 18:59:11.339955 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:11.339961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:11.340151 (+ 190us) server_negotiation.cc:407] Connection header received
0812 18:59:11.340300 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:11.340302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:11.340352 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:11.340462 (+ 110us) server_negotiation.cc:226] Negotiated authn=JWT
0812 18:59:11.340767 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.341299 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.342532 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.342778 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.342866 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:11.343183 (+ 317us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 18:59:11.344566 (+ 1383us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35065/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 18:59:11.344787 (+ 221us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50334: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35065/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 18:59:11.345355 4140 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50334: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35065/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 18:59:11.347150 4168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:11.339735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:41479 (local address 127.0.0.1:50334)
0812 18:59:11.340048 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:11.340063 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:11.340173 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:11.340476 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:11.340479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:11.340487 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 18:59:11.340605 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:11.340611 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.341412 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.341415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:11.342405 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:11.342415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.342897 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.342901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:11.342986 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:11.343014 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 18:59:11.344725 (+ 1711us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35065/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 18:59:11.346975 (+ 2250us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.112.254:41479: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35065/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":2070,"thread_start_us":151,"threads_started":1}
I20260812 18:59:11.347723 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 4055
E20260812 18:59:11.358621 4169 webserver.cc:533] Webserver: sslize failed:
E20260812 18:59:11.358692 4169 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T18:59:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (315 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T18:59:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:11Z Disabled control of system clock
I20260812 18:59:11.403112 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:42793
--webserver_interface=127.29.112.254
--webserver_port=0
--builtin_ntp_servers=127.29.112.212:36733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:42793
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/testchainca.cert with env {}
W20260812 18:59:11.513386 4173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:11.513715 4173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:11.513868 4173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:11.517975 4173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:11.518055 4173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:11.518085 4173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:11.518126 4173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:11.518164 4173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 18:59:11.518203 4173 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 18:59:11.518234 4173 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 18:59:11.523514 4173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:42793
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:42793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:11.524559 4173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:11.525588 4173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:11.534173 4179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:11.534180 4181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:11.534217 4178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:11.534385 4173 server_base.cc:1034] running on GCE node
I20260812 18:59:11.534989 4173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:11.536114 4173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:11.537309 4173 hybrid_clock.cc:648] HybridClock initialized: now 1786561151537294 us; error 26 us; skew 500 ppm
I20260812 18:59:11.539157 4173 webserver.cc:460] Webserver started at http://127.29.112.254:46227/ using document root <none> and password file <none>
I20260812 18:59:11.539726 4173 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:11.539819 4173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:11.540056 4173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:11.542131 4173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "696d54e94b65463b90b4fb83abc82468"
format_stamp: "Formatted at 2026-08-12 18:59:11 on dist-test-slave-6nmv"
server_key: "4cf83b05eaa5d0b134d5fd9fdf2ecff6"
server_key_iv: "183a28e3569503d8ec0a6992a9a8dc58"
server_key_version: "encryptionkey@0"
I20260812 18:59:11.542711 4173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "696d54e94b65463b90b4fb83abc82468"
format_stamp: "Formatted at 2026-08-12 18:59:11 on dist-test-slave-6nmv"
server_key: "4cf83b05eaa5d0b134d5fd9fdf2ecff6"
server_key_iv: "183a28e3569503d8ec0a6992a9a8dc58"
server_key_version: "encryptionkey@0"
I20260812 18:59:11.546264 4173 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260812 18:59:11.548659 4187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:11.549791 4173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 18:59:11.549981 4173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "696d54e94b65463b90b4fb83abc82468"
format_stamp: "Formatted at 2026-08-12 18:59:11 on dist-test-slave-6nmv"
server_key: "4cf83b05eaa5d0b134d5fd9fdf2ecff6"
server_key_iv: "183a28e3569503d8ec0a6992a9a8dc58"
server_key_version: "encryptionkey@0"
I20260812 18:59:11.550103 4173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:11.567468 4173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:11.568120 4173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:11.568305 4173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:11.574972 4173 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:42793
I20260812 18:59:11.574990 4248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:42793 every 8 connection(s)
I20260812 18:59:11.575845 4173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:11.578572 4249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:11.578963 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 4173
I20260812 18:59:11.579079 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:11.579388 30147 external_mini_cluster.cc:1428] Setting key 66d2112fc08ffa9b1effd7b5f504e5dc
I20260812 18:59:11.584767 4249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468: Bootstrap starting.
I20260812 18:59:11.587080 4249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:11.587787 4249 log.cc:824] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:11.588171 4252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:11.580664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56528 (local address 127.29.112.254:42793)
0812 18:59:11.581229 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:11.581246 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:11.581278 (+ 32us) server_negotiation.cc:407] Connection header received
0812 18:59:11.581954 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:11.581971 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:11.582335 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:11.582680 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:11.583052 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.583912 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.584749 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.585220 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.585379 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:11.586349 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:11.586353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:11.586364 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:11.586743 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:11.586801 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:11.586818 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:11.586918 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:11.587081 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 18:59:11.587434 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":131,"threads_started":1}
I20260812 18:59:11.589766 4249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468: No bootstrap required, opened a new log
I20260812 18:59:11.590922 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 18:59:11.593680 4249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696d54e94b65463b90b4fb83abc82468" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42793 } }
I20260812 18:59:11.593937 4249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:11.594000 4249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696d54e94b65463b90b4fb83abc82468, State: Initialized, Role: FOLLOWER
I20260812 18:59:11.594347 4249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696d54e94b65463b90b4fb83abc82468" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42793 } }
I20260812 18:59:11.594492 4249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:11.594551 4249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:11.594635 4249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:11.595568 4249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696d54e94b65463b90b4fb83abc82468" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42793 } }
I20260812 18:59:11.595890 4249 leader_election.cc:302] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 696d54e94b65463b90b4fb83abc82468; no voters:
I20260812 18:59:11.596171 4249 leader_election.cc:288] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:11.596261 4254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:11.596450 4254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [term 1 LEADER]: Becoming Leader. State: Replica: 696d54e94b65463b90b4fb83abc82468, State: Running, Role: LEADER
I20260812 18:59:11.596763 4254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696d54e94b65463b90b4fb83abc82468" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42793 } }
I20260812 18:59:11.597347 4249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:11.598278 4255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 696d54e94b65463b90b4fb83abc82468. Latest consensus state: current_term: 1 leader_uuid: "696d54e94b65463b90b4fb83abc82468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696d54e94b65463b90b4fb83abc82468" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42793 } } }
I20260812 18:59:11.598409 4255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:11.598677 4256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "696d54e94b65463b90b4fb83abc82468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696d54e94b65463b90b4fb83abc82468" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42793 } } }
I20260812 18:59:11.598737 4265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:11.598765 4256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:11.601415 4265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:11.605798 4265 catalog_manager.cc:1347] Generated new cluster ID: 8d81c8e35bcc4537a8d709d74dce9fd1
I20260812 18:59:11.605883 4265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:11.619601 4265 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:11.620434 4265 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:11.625613 4265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 696d54e94b65463b90b4fb83abc82468: Generated new TSK 0
I20260812 18:59:11.626283 4265 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 18:59:12.138274 4252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:11.640436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.29.112.254:42793)
0812 18:59:11.640565 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:11.640570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:11.640719 (+ 149us) server_negotiation.cc:407] Connection header received
0812 18:59:11.640838 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:11.640841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:11.640901 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:11.641030 (+ 129us) server_negotiation.cc:226] Negotiated authn=JWT
0812 18:59:11.641251 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.642077 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.642832 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.643166 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.643261 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:11.643511 (+ 250us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 18:59:12.137736 (+494225us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 18:59:12.138121 (+ 385us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56540: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 18:59:12.138360 4283 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:11.640316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:42793 (local address 127.0.0.1:56540)
0812 18:59:11.640634 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:11.640647 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:11.640738 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:11.641045 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:11.641047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:11.641053 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0812 18:59:11.641145 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:11.641149 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.642180 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.642182 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:11.642728 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:11.642734 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:11.643279 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:11.643281 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:11.643343 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:11.643363 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 18:59:12.138001 (+494638us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 18:59:12.138191 (+ 190us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.112.254:42793: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":130,"threads_started":1}
W20260812 18:59:12.138367 4252 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56540: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260812 18:59:12.139030 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 4173
2026-08-12T18:59:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (770 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (118 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (122 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:59:12 dist-test-slave-6nmv krb5kdc[4296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:59:12 dist-test-slave-6nmv krb5kdc[4296](info): set up 2 sockets
Aug 12 18:59:12 dist-test-slave-6nmv krb5kdc[4296](info): commencing operation
krb5kdc: starting...
W20260812 18:59:13.483940 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:13 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:59:13.609028 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:45393
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:35627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:45393
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:13.711956 4309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:13.712208 4309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:13.712248 4309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:13.716931 4309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:13.716985 4309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:13.717006 4309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:13.717057 4309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:13.717092 4309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:13.722173 4309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:35627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:45393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:45393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:13.723125 4309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:13.724071 4309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:13.732431 4316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:13.732417 4319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:13.732551 4309 server_base.cc:1034] running on GCE node
W20260812 18:59:13.732417 4315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:13.733204 4309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:13.734370 4309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:13.735536 4309 hybrid_clock.cc:648] HybridClock initialized: now 1786561153735520 us; error 32 us; skew 500 ppm
Aug 12 18:59:13 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:13.738431 4309 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:59:13.739293 4309 webserver.cc:460] Webserver started at http://127.29.112.254:44175/ using document root <none> and password file <none>
I20260812 18:59:13.739830 4309 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:13.739889 4309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:13.740113 4309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:13.742137 4309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "a9d96157a3e34498ada7666ff11e3bd0"
format_stamp: "Formatted at 2026-08-12 18:59:13 on dist-test-slave-6nmv"
server_key: "d71bdd669472e0fe7f2991531c856310"
server_key_iv: "e7720802e9fad88b3126189f3f7b02f3"
server_key_version: "encryptionkey@0"
I20260812 18:59:13.742671 4309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "a9d96157a3e34498ada7666ff11e3bd0"
format_stamp: "Formatted at 2026-08-12 18:59:13 on dist-test-slave-6nmv"
server_key: "d71bdd669472e0fe7f2991531c856310"
server_key_iv: "e7720802e9fad88b3126189f3f7b02f3"
server_key_version: "encryptionkey@0"
I20260812 18:59:13.746199 4309 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 18:59:13.748481 4326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:13.749581 4309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:13.749716 4309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "a9d96157a3e34498ada7666ff11e3bd0"
format_stamp: "Formatted at 2026-08-12 18:59:13 on dist-test-slave-6nmv"
server_key: "d71bdd669472e0fe7f2991531c856310"
server_key_iv: "e7720802e9fad88b3126189f3f7b02f3"
server_key_version: "encryptionkey@0"
I20260812 18:59:13.749840 4309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:13.765630 4309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:13.768508 4309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:13.768700 4309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:13.775453 4309 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:45393
I20260812 18:59:13.775476 4386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:45393 every 8 connection(s)
I20260812 18:59:13.776381 4309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:13.779057 4388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:13.784125 4388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0: Bootstrap starting.
I20260812 18:59:13.784901 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 4309
I20260812 18:59:13.785033 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:13.785348 30147 external_mini_cluster.cc:1428] Setting key fd31f74cbe58cad45503bb7936af493a
I20260812 18:59:13.787222 4388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:13.787982 4388 log.cc:824] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:13.789741 4388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0: No bootstrap required, opened a new log
I20260812 18:59:13.793282 4388 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d96157a3e34498ada7666ff11e3bd0" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45393 } }
I20260812 18:59:13.793541 4388 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:13.793607 4388 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9d96157a3e34498ada7666ff11e3bd0, State: Initialized, Role: FOLLOWER
Aug 12 18:59:13 dist-test-slave-6nmv krb5kdc[4296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:13.794106 4388 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d96157a3e34498ada7666ff11e3bd0" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45393 } }
I20260812 18:59:13.794273 4388 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:13.794343 4388 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:13.794458 4388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:13.795507 4388 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d96157a3e34498ada7666ff11e3bd0" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45393 } }
I20260812 18:59:13.795866 4388 leader_election.cc:302] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9d96157a3e34498ada7666ff11e3bd0; no voters:
I20260812 18:59:13.796183 4388 leader_election.cc:288] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:13.796280 4393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:13.796492 4393 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [term 1 LEADER]: Becoming Leader. State: Replica: a9d96157a3e34498ada7666ff11e3bd0, State: Running, Role: LEADER
I20260812 18:59:13.796885 4393 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d96157a3e34498ada7666ff11e3bd0" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45393 } }
I20260812 18:59:13.797610 4388 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:13.798130 4393 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9d96157a3e34498ada7666ff11e3bd0. Latest consensus state: current_term: 1 leader_uuid: "a9d96157a3e34498ada7666ff11e3bd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d96157a3e34498ada7666ff11e3bd0" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45393 } } }
I20260812 18:59:13.798895 4393 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:13.798779 4394 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9d96157a3e34498ada7666ff11e3bd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d96157a3e34498ada7666ff11e3bd0" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45393 } } }
I20260812 18:59:13.799204 4394 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:13.799495 4403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:13.800261 4391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:13.786574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.29.112.254:45393)
0812 18:59:13.787038 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:13.787053 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:13.787086 (+ 33us) server_negotiation.cc:407] Connection header received
0812 18:59:13.787753 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:13.787771 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:13.788140 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:13.788467 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:13.789269 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:13.790256 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:13.791097 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:13.791566 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:13.791773 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:13.794542 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:13.794550 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:13.794577 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:13.794605 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:13.796447 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:13.796964 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:13.796969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:13.796974 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:13.797109 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:13.797447 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:13.797450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:13.797452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:13.798003 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:13.798809 (+ 806us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:13.798995 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 18:59:13.799299 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":121,"threads_started":1}
I20260812 18:59:13.802548 4403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:13.807783 4403 catalog_manager.cc:1347] Generated new cluster ID: fb44a0ed550f45cd84f2a4d84636c968
I20260812 18:59:13.807844 4403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:13.816846 4403 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:13.817709 4403 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:13.821688 4403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9d96157a3e34498ada7666ff11e3bd0: Generated new TSK 0
I20260812 18:59:13.822369 4403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 18:59:13.854910 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:45393
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:35627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:13.961396 4418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:13.961663 4418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:13.961699 4418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:13.965240 4418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:13.965284 4418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:13.965372 4418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:13.970126 4418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:35627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:45393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:13.971009 4418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:13.971894 4418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:13.980929 4423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:13.981142 4418 server_base.cc:1034] running on GCE node
W20260812 18:59:13.981235 4424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:13.981020 4426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:13.981722 4418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:13.982448 4418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:13.983623 4418 hybrid_clock.cc:648] HybridClock initialized: now 1786561153983598 us; error 40 us; skew 500 ppm
Aug 12 18:59:13 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:13.991127 4418 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 18:59:13.992087 4418 webserver.cc:460] Webserver started at http://127.29.112.193:35751/ using document root <none> and password file <none>
I20260812 18:59:13.992655 4418 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:13.992754 4418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:13.993009 4418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:13.995150 4418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "c2d46af471ab489f9fbb0098a9f26366"
format_stamp: "Formatted at 2026-08-12 18:59:13 on dist-test-slave-6nmv"
server_key: "eb3b96b9dce422088dd03be9019f8f98"
server_key_iv: "e34caf29762c1568e4a8856f055b66e0"
server_key_version: "encryptionkey@0"
I20260812 18:59:13.995761 4418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "c2d46af471ab489f9fbb0098a9f26366"
format_stamp: "Formatted at 2026-08-12 18:59:13 on dist-test-slave-6nmv"
server_key: "eb3b96b9dce422088dd03be9019f8f98"
server_key_iv: "e34caf29762c1568e4a8856f055b66e0"
server_key_version: "encryptionkey@0"
I20260812 18:59:13.999666 4418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 18:59:14.002060 4434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:14.003167 4418 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 18:59:14.003304 4418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "c2d46af471ab489f9fbb0098a9f26366"
format_stamp: "Formatted at 2026-08-12 18:59:13 on dist-test-slave-6nmv"
server_key: "eb3b96b9dce422088dd03be9019f8f98"
server_key_iv: "e34caf29762c1568e4a8856f055b66e0"
server_key_version: "encryptionkey@0"
I20260812 18:59:14.003413 4418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:14.027869 4418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:14.030792 4418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:14.030988 4418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:14.031553 4418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:14.032397 4418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:14.032477 4418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:14.032584 4418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:14.032635 4418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:14.041252 4418 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:42537
I20260812 18:59:14.041281 4565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:42537 every 8 connection(s)
I20260812 18:59:14.042227 4418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:14.051748 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 4418
I20260812 18:59:14.051879 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:14.052242 30147 external_mini_cluster.cc:1428] Setting key c111bc93f6ce0822a7fa11c32bb5a5b2
I20260812 18:59:14.056031 4391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:14.043899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:55207 (local address 127.29.112.254:45393)
0812 18:59:14.044045 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:14.044048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:14.044765 (+ 717us) server_negotiation.cc:407] Connection header received
0812 18:59:14.046135 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:14.046138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:14.046198 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:14.046305 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:14.047455 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.047983 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.049144 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.049391 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.049559 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:14.052102 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:14.052107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:14.052110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:14.052140 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:14.053977 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.054421 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.054427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.054431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.054542 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.054852 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.054858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.054862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.055104 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:14.055205 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:14.055617 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 18:59:14.055782 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 18:59:14.056857 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:14.044207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:45393 (local address 127.29.112.193:55207)
0812 18:59:14.044619 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:14.044659 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:14.045905 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:14.046476 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:14.046489 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:14.046833 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:14.047299 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:14.047312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.048117 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.048122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:14.048998 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:14.049008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.049610 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.049614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:14.049770 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:14.050389 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:14.050403 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:14.051910 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:14.054076 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:14.054079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:14.054089 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:14.054316 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:14.054635 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:14.054637 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:14.054638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:14.054766 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:14.055200 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:14.055204 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:14.055399 (+ 195us) client_negotiation.cc:769] Sending connection context
0812 18:59:14.055543 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 18:59:14.055810 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":109,"threads_started":1}
I20260812 18:59:14.058121 4566 heartbeater.cc:347] Connected to a master server at 127.29.112.254:45393
I20260812 18:59:14.058380 4566 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:14.058938 4566 heartbeater.cc:511] Master 127.29.112.254:45393 requested a full tablet report, sending...
I20260812 18:59:14.060449 4347 ts_manager.cc:194] Registered new tserver with Master: c2d46af471ab489f9fbb0098a9f26366 (127.29.112.193:42537)
I20260812 18:59:14.061546 4347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:55207
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 18:59:14.080235 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:45393
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:35627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:14.185314 4573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:14.185552 4573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:14.185635 4573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:14.189519 4573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:14.189594 4573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:14.189684 4573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:14.194543 4573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:35627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:45393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:14.195513 4573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:14.196465 4573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:14.205224 4581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:14.205346 4583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:14.205224 4580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:14.205646 4573 server_base.cc:1034] running on GCE node
I20260812 18:59:14.206076 4573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:14.206686 4573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:14.207872 4573 hybrid_clock.cc:648] HybridClock initialized: now 1786561154207837 us; error 52 us; skew 500 ppm
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:14.210659 4573 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 18:59:14.211503 4573 webserver.cc:460] Webserver started at http://127.29.112.194:39235/ using document root <none> and password file <none>
I20260812 18:59:14.212070 4573 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:14.212163 4573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:14.212393 4573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:14.214509 4573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "279bfd8c6cbc43569decd31d02099953"
format_stamp: "Formatted at 2026-08-12 18:59:14 on dist-test-slave-6nmv"
server_key: "cf1c743b0179cf5186d6370f1bc3113c"
server_key_iv: "6aece96a7dd5b1ee3789f9e221aa0eaf"
server_key_version: "encryptionkey@0"
I20260812 18:59:14.215113 4573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "279bfd8c6cbc43569decd31d02099953"
format_stamp: "Formatted at 2026-08-12 18:59:14 on dist-test-slave-6nmv"
server_key: "cf1c743b0179cf5186d6370f1bc3113c"
server_key_iv: "6aece96a7dd5b1ee3789f9e221aa0eaf"
server_key_version: "encryptionkey@0"
I20260812 18:59:14.218613 4573 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 18:59:14.220862 4592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:14.222034 4573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 18:59:14.222157 4573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "279bfd8c6cbc43569decd31d02099953"
format_stamp: "Formatted at 2026-08-12 18:59:14 on dist-test-slave-6nmv"
server_key: "cf1c743b0179cf5186d6370f1bc3113c"
server_key_iv: "6aece96a7dd5b1ee3789f9e221aa0eaf"
server_key_version: "encryptionkey@0"
I20260812 18:59:14.222270 4573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:14.240595 4573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:14.243743 4573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:14.243947 4573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:14.244508 4573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:14.245263 4573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:14.245342 4573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:14.245421 4573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:14.245455 4573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:14.254197 4573 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:42029
I20260812 18:59:14.254233 4720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:42029 every 8 connection(s)
I20260812 18:59:14.254977 4573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:14.255805 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 4573
I20260812 18:59:14.255944 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:14.256214 30147 external_mini_cluster.cc:1428] Setting key e5365e112b53e57bacfc1d2531e93b16
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
I20260812 18:59:14.270186 4391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:14.257664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:58857 (local address 127.29.112.254:45393)
0812 18:59:14.257798 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:14.257846 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:14.258745 (+ 899us) server_negotiation.cc:407] Connection header received
0812 18:59:14.260037 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:14.260040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:14.260089 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:14.260166 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:14.261395 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.261964 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.263741 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.263936 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.264017 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:14.266804 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:14.266807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:14.266809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:14.266832 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:14.268278 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.268748 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.268752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.268754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.268832 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.269145 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.269149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.269151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.269314 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:14.269415 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:14.269894 (+ 479us) server_negotiation.cc:299] Negotiation successful
0812 18:59:14.270020 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 18:59:14.271091 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:14.258156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:45393 (local address 127.29.112.194:58857)
0812 18:59:14.258621 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:14.258657 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:14.259852 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:14.260334 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:14.260340 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:14.260648 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:14.261246 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:14.261266 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.262100 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.262107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:14.263580 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:14.263593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.264079 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.264084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:14.264298 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:14.264918 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:14.264938 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:14.266635 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:14.268393 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:14.268398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:14.268410 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:14.268650 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:14.268920 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:14.268923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:14.268925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:14.269055 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:14.269414 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:14.269420 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:14.269640 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 18:59:14.269852 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 18:59:14.270140 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":139,"threads_started":1}
I20260812 18:59:14.272255 4721 heartbeater.cc:347] Connected to a master server at 127.29.112.254:45393
I20260812 18:59:14.272507 4721 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:14.273099 4721 heartbeater.cc:511] Master 127.29.112.254:45393 requested a full tablet report, sending...
I20260812 18:59:14.274163 4347 ts_manager.cc:194] Registered new tserver with Master: 279bfd8c6cbc43569decd31d02099953 (127.29.112.194:42029)
I20260812 18:59:14.274706 4347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:58857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 18:59:14.288472 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:45393
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:35627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:14.395509 4727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:14.395813 4727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:14.395886 4727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:14.399865 4727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:14.399935 4727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:14.400018 4727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:14.404650 4727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:35627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:45393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:14.405639 4727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:14.406625 4727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:14.415935 4737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:14.415941 4734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:14.415972 4735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:14.416116 4727 server_base.cc:1034] running on GCE node
I20260812 18:59:14.416575 4727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:14.417189 4727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:14.418345 4727 hybrid_clock.cc:648] HybridClock initialized: now 1786561154418311 us; error 49 us; skew 500 ppm
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:14.421260 4727 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 18:59:14.422399 4727 webserver.cc:460] Webserver started at http://127.29.112.195:33929/ using document root <none> and password file <none>
I20260812 18:59:14.422950 4727 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:14.423053 4727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:14.423249 4727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:14.425436 4727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "6b4617193a7f4649b8d783cf8138e0d5"
format_stamp: "Formatted at 2026-08-12 18:59:14 on dist-test-slave-6nmv"
server_key: "6bf45623add122cd09caf7c6af82cbe3"
server_key_iv: "2fc187f5ba1ad68ca0e24f56e0acf173"
server_key_version: "encryptionkey@0"
I20260812 18:59:14.426203 4727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "6b4617193a7f4649b8d783cf8138e0d5"
format_stamp: "Formatted at 2026-08-12 18:59:14 on dist-test-slave-6nmv"
server_key: "6bf45623add122cd09caf7c6af82cbe3"
server_key_iv: "2fc187f5ba1ad68ca0e24f56e0acf173"
server_key_version: "encryptionkey@0"
I20260812 18:59:14.429939 4727 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 18:59:14.432308 4744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:14.433540 4727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:14.433698 4727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "6b4617193a7f4649b8d783cf8138e0d5"
format_stamp: "Formatted at 2026-08-12 18:59:14 on dist-test-slave-6nmv"
server_key: "6bf45623add122cd09caf7c6af82cbe3"
server_key_iv: "2fc187f5ba1ad68ca0e24f56e0acf173"
server_key_version: "encryptionkey@0"
I20260812 18:59:14.433810 4727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:14.453991 4727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:14.456859 4727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:14.457087 4727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:14.457660 4727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:14.458528 4727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:14.458601 4727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:14.458670 4727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:14.458720 4727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:14.466933 4727 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:45787
I20260812 18:59:14.466980 4875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:45787 every 8 connection(s)
I20260812 18:59:14.467777 4727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:59:14.474913 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 4727
I20260812 18:59:14.475035 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:14.475276 30147 external_mini_cluster.cc:1428] Setting key 41de7c0987fb08e723e0ddec85a8e1c9
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:14.480841 4391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:14.469528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:46515 (local address 127.29.112.254:45393)
0812 18:59:14.469674 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:14.469677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:14.470416 (+ 739us) server_negotiation.cc:407] Connection header received
0812 18:59:14.471799 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:14.471802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:14.471862 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:14.471976 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:14.473102 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.473590 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.474624 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.474854 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.474964 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:14.477684 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:14.477687 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:14.477689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:14.477711 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:14.479026 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.479457 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.479459 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.479460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.479530 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.479838 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.479840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.479842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.480015 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:14.480089 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:14.480648 (+ 559us) server_negotiation.cc:299] Negotiation successful
0812 18:59:14.480733 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 18:59:14.481572 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:14.469791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:45393 (local address 127.29.112.195:46515)
0812 18:59:14.470274 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:14.470311 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:14.471567 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:14.472150 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:14.472156 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:14.472502 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:14.472966 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:14.472978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.473706 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.473709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:14.474497 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:14.474507 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.475001 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.475005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:14.475203 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:14.475829 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:14.475852 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:14.477521 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:14.479123 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:14.479127 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:14.479136 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:14.479376 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:14.479608 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:14.479610 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:14.479612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:14.479735 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:14.480214 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:14.480219 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:14.480439 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 18:59:14.480605 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 18:59:14.480825 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":103,"threads_started":1}
I20260812 18:59:14.482650 4876 heartbeater.cc:347] Connected to a master server at 127.29.112.254:45393
I20260812 18:59:14.482839 4876 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:14.483264 4876 heartbeater.cc:511] Master 127.29.112.254:45393 requested a full tablet report, sending...
I20260812 18:59:14.484210 4347 ts_manager.cc:194] Registered new tserver with Master: 6b4617193a7f4649b8d783cf8138e0d5 (127.29.112.195:45787)
I20260812 18:59:14.484762 4347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:46515
I20260812 18:59:14.489161 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/test-admin.keytab.
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:14.507253 30147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 18:59:14.509667 4883 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:14.508463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41414 (local address 127.29.112.193:42537)
0812 18:59:14.508726 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:14.508739 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:14.508761 (+ 22us) server_negotiation.cc:407] Connection header received
0812 18:59:14.508804 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:14.508808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:14.508983 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:14.509088 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:14.509463 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41414: BlockingRecv error: recv got EOF from 127.0.0.1:41414 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":86,"threads_started":1}
W20260812 18:59:14.509722 30147 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/krb5cc' of type FILE
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:14.510645 30147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:14.511479 30147 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.193@KRBTEST.COM
I20260812 18:59:14.520601 4883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:14.512245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41430 (local address 127.29.112.193:42537)
0812 18:59:14.512373 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:14.512377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:14.512480 (+ 103us) server_negotiation.cc:407] Connection header received
0812 18:59:14.512590 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:14.512594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:14.512636 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:14.512718 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:14.513212 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.513950 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.514752 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.515107 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.515219 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:14.517568 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:14.517576 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:14.517586 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:14.517607 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:14.519206 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.519581 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.519585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.519586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.519668 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.519884 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.519886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.519887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.520100 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:14.520235 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:14.520376 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 18:59:14.520482 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 18:59:14.523773 4883 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:14.522935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.29.112.193:42537)
0812 18:59:14.523051 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:14.523054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:14.523154 (+ 100us) server_negotiation.cc:407] Connection header received
0812 18:59:14.523251 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:14.523253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:14.523289 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:14.523360 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:14.523707 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45046: BlockingRecv error: recv got EOF from 127.0.0.1:45046 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 18:59:14.523967 30147 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/krb5cc' of type FILE
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:14.524879 30147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:14.525758 30147 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.193@KRBTEST.COM
I20260812 18:59:14.533494 4883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:14.526656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.29.112.193:42537)
0812 18:59:14.526774 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:14.526778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:14.526893 (+ 115us) server_negotiation.cc:407] Connection header received
0812 18:59:14.527008 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:14.527012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:14.527064 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:14.527146 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:14.527598 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.528041 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.528747 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.528947 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.529042 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:14.530972 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:14.530975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:14.530977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:14.530995 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:14.532130 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.532530 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.532532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.532533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.532604 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.532871 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.532874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.532875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.533035 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:14.533134 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:14.533310 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 18:59:14.533394 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 18:59:14.535867 4883 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:14.534928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.29.112.193:42537)
0812 18:59:14.535032 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:14.535036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:14.535145 (+ 109us) server_negotiation.cc:407] Connection header received
0812 18:59:14.535253 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:14.535256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:14.535293 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:14.535367 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:14.535776 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45058: BlockingRecv error: recv got EOF from 127.0.0.1:45058 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 18:59:14.536027 30147 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786561125017040-30147-0/krb5kdc/krb5cc' of type FILE
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:14.536974 30147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:14.537896 30147 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.193@KRBTEST.COM
I20260812 18:59:14.545681 4883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:14.538576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45070 (local address 127.29.112.193:42537)
0812 18:59:14.538675 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:14.538681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:14.538775 (+ 94us) server_negotiation.cc:407] Connection header received
0812 18:59:14.538868 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:14.538870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:14.538915 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:14.538981 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:14.539440 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.539897 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.540662 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:14.540863 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:14.540959 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:14.543083 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:14.543086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:14.543087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:14.543107 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:14.544242 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.544600 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.544603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.544604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.544682 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:14.544957 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:14.544959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:14.544961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:14.545106 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:14.545202 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:14.545377 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 18:59:14.545524 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 18:59:14.546622 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 4418
I20260812 18:59:14.556743 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 4573
I20260812 18:59:14.563356 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 4727
I20260812 18:59:14.570231 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 4309
2026-08-12T18:59:14Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2189 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4898](info): set up 2 sockets
Aug 12 18:59:14 dist-test-slave-6nmv krb5kdc[4898](info): commencing operation
krb5kdc: starting...
W20260812 18:59:15.676564 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:15 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.254.keytab.
Entry for principal oryx/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.254.keytab.
I20260812 18:59:15.791057 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:40813
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.254.keytab
--principal=oryx/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:40813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:15.897289 4912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:15.897593 4912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:15.897679 4912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:15.901367 4912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:15.901451 4912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:15.901487 4912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:15.901542 4912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:15.901577 4912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:15.906349 4912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:40813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.254.keytab
--principal=oryx/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:40813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:15.907400 4912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:15.908423 4912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:15.916623 4922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:15.916611 4920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:15.916623 4918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:15.916921 4912 server_base.cc:1034] running on GCE node
I20260812 18:59:15.917575 4912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:15.918632 4912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:15.919807 4912 hybrid_clock.cc:648] HybridClock initialized: now 1786561155919797 us; error 29 us; skew 500 ppm
Aug 12 18:59:15 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:15.922542 4912 init.cc:377] Logged in from keytab as oryx/127.29.112.254@KRBTEST.COM (short username oryx)
I20260812 18:59:15.923312 4912 webserver.cc:460] Webserver started at http://127.29.112.254:43779/ using document root <none> and password file <none>
I20260812 18:59:15.923866 4912 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:15.923952 4912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:15.924196 4912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:15.926216 4912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "d5d24a92401844d588331250ab11bd13"
format_stamp: "Formatted at 2026-08-12 18:59:15 on dist-test-slave-6nmv"
server_key: "9265b07305752f3a14dee63f0abf5b81"
server_key_iv: "b9dc8c34d5d8b87a257e75f205527c1a"
server_key_version: "encryptionkey@0"
I20260812 18:59:15.926815 4912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "d5d24a92401844d588331250ab11bd13"
format_stamp: "Formatted at 2026-08-12 18:59:15 on dist-test-slave-6nmv"
server_key: "9265b07305752f3a14dee63f0abf5b81"
server_key_iv: "b9dc8c34d5d8b87a257e75f205527c1a"
server_key_version: "encryptionkey@0"
I20260812 18:59:15.930269 4912 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 18:59:15.932597 4931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:15.933748 4912 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:15.933907 4912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "d5d24a92401844d588331250ab11bd13"
format_stamp: "Formatted at 2026-08-12 18:59:15 on dist-test-slave-6nmv"
server_key: "9265b07305752f3a14dee63f0abf5b81"
server_key_iv: "b9dc8c34d5d8b87a257e75f205527c1a"
server_key_version: "encryptionkey@0"
I20260812 18:59:15.934013 4912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:15.949055 4912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:15.952097 4912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:15.952306 4912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:15.959115 4912 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:40813
I20260812 18:59:15.959188 4994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:40813 every 8 connection(s)
I20260812 18:59:15.960070 4912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:15.962874 4995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:15.967133 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 4912
I20260812 18:59:15.967273 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:15.967536 30147 external_mini_cluster.cc:1428] Setting key b84f9a592f5f05103ef4cc15209571ab
I20260812 18:59:15.968631 4995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13: Bootstrap starting.
I20260812 18:59:15.971266 4995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:15.972271 4995 log.cc:824] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:15.974267 4995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13: No bootstrap required, opened a new log
Aug 12 18:59:15 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:15.978295 4995 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } }
I20260812 18:59:15.978533 4995 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:15.978597 4995 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d24a92401844d588331250ab11bd13, State: Initialized, Role: FOLLOWER
I20260812 18:59:15.979035 4995 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } }
I20260812 18:59:15.979154 4995 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:15.979241 4995 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:15.979367 4995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:15.980594 4995 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } }
I20260812 18:59:15.980962 4995 leader_election.cc:302] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5d24a92401844d588331250ab11bd13; no voters:
I20260812 18:59:15.981367 4995 leader_election.cc:288] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:15.981459 5000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:15.981730 5000 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 1 LEADER]: Becoming Leader. State: Replica: d5d24a92401844d588331250ab11bd13, State: Running, Role: LEADER
I20260812 18:59:15.982089 5000 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } }
I20260812 18:59:15.982663 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:15.968852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.29.112.254:40813)
0812 18:59:15.969246 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:15.969266 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:15.969305 (+ 39us) server_negotiation.cc:407] Connection header received
0812 18:59:15.970134 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:15.970172 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:15.970554 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:15.970887 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:15.971586 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:15.973041 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:15.973858 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:15.974510 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:15.974689 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:15.977505 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:15.977511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:15.977523 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:15.977546 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:15.979499 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:15.980016 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:15.980020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:15.980024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:15.980129 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:15.980472 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:15.980474 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:15.980475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:15.980896 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:15.981043 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:15.981431 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 18:59:15.981727 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":111,"threads_started":1}
I20260812 18:59:15.982677 4995 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:15.983727 5002 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } }
I20260812 18:59:15.983843 5002 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:15.983959 5003 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5d24a92401844d588331250ab11bd13. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } }
I20260812 18:59:15.984112 5003 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: This master's current role is: LEADER
W20260812 18:59:15.987682 5018 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 18:59:15.987761 5018 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 18:59:15.987838 5012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:15.988646 5012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:15.993786 5012 catalog_manager.cc:1347] Generated new cluster ID: 85d01156aff54b33b86e23f078b6ba6e
I20260812 18:59:15.993943 5012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:16.004637 5012 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:16.005533 5012 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:16.010092 5012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13: Generated new TSK 0
I20260812 18:59:16.010762 5012 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab.
Entry for principal oryx/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab.
I20260812 18:59:16.049520 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:40813
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab
--principal=oryx/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:16.199290 5024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:16.199697 5024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:16.199810 5024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:16.205394 5024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:16.205531 5024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:16.205653 5024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:16.212369 5024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab
--principal=oryx/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:40813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:16.213426 5024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:16.214589 5024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:16.223898 5029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:16.224031 5024 server_base.cc:1034] running on GCE node
W20260812 18:59:16.224038 5032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:16.224275 5030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:16.224644 5024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:16.225288 5024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:16.226466 5024 hybrid_clock.cc:648] HybridClock initialized: now 1786561156226448 us; error 30 us; skew 500 ppm
Aug 12 18:59:16 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:16.229245 5024 init.cc:377] Logged in from keytab as oryx/127.29.112.193@KRBTEST.COM (short username oryx)
I20260812 18:59:16.230160 5024 webserver.cc:460] Webserver started at http://127.29.112.193:37477/ using document root <none> and password file <none>
I20260812 18:59:16.230731 5024 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:16.230834 5024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:16.231076 5024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:16.233074 5024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "17baa51a90904d65b6a9a07958887c47"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "3ed7ee697ccee0870e5aa493e6fa0b0e"
server_key_iv: "1a1fbac72f1ba25358acd16bb0a21565"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.233647 5024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "17baa51a90904d65b6a9a07958887c47"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "3ed7ee697ccee0870e5aa493e6fa0b0e"
server_key_iv: "1a1fbac72f1ba25358acd16bb0a21565"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.237468 5024 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 18:59:16.240054 5041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:16.241281 5024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:16.241418 5024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "17baa51a90904d65b6a9a07958887c47"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "3ed7ee697ccee0870e5aa493e6fa0b0e"
server_key_iv: "1a1fbac72f1ba25358acd16bb0a21565"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.241541 5024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:16.257624 5024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:16.260823 5024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:16.261016 5024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:16.261570 5024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:16.262392 5024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:16.262461 5024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:16.262529 5024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:16.262571 5024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:16.271552 5024 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:41303
I20260812 18:59:16.271587 5167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:41303 every 8 connection(s)
I20260812 18:59:16.272440 5024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:16.277916 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 5024
I20260812 18:59:16.278040 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:16.278353 30147 external_mini_cluster.cc:1428] Setting key 14fdc44356e4caad24708eb9ccd02124
Aug 12 18:59:16 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:16.287899 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.274372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:59345 (local address 127.29.112.254:40813)
0812 18:59:16.274512 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:16.274516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:16.275315 (+ 799us) server_negotiation.cc:407] Connection header received
0812 18:59:16.276704 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:16.276707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:16.276769 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:16.276874 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:16.278187 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.278844 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.280316 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.280540 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.280621 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.283542 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:16.283549 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:16.283555 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:16.283588 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:16.285605 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.286216 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.286221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.286225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.286329 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.286685 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.286692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.286694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.286948 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:16.287068 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:16.287523 (+ 455us) server_negotiation.cc:299] Negotiation successful
0812 18:59:16.287684 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 18:59:16.288509 5171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.274716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.193:59345)
0812 18:59:16.275163 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:16.275203 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:16.276482 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:16.277062 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:16.277069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:16.277417 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:16.277970 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.278005 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.278990 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.278993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.280182 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.280194 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.280695 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.280702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.280934 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.281553 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:16.281579 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:16.283375 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:16.285738 (+ 2363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.285743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.285754 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.286114 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.286441 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.286443 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.286444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.286575 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.287056 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:16.287061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:16.287273 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 18:59:16.287429 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 18:59:16.287676 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":131,"threads_started":1}
WARNING: no policy specified for oryx/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 18:59:16.290019 5168 heartbeater.cc:347] Connected to a master server at 127.29.112.254:40813
I20260812 18:59:16.290251 5168 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:16.290720 5168 heartbeater.cc:511] Master 127.29.112.254:40813 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.194@KRBTEST.COM" created.
I20260812 18:59:16.292358 4955 ts_manager.cc:194] Registered new tserver with Master: 17baa51a90904d65b6a9a07958887c47 (127.29.112.193:41303)
I20260812 18:59:16.294034 4955 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:59345
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab.
Entry for principal oryx/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab.
I20260812 18:59:16.307010 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:40813
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab
--principal=oryx/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:16.414812 5174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:16.415067 5174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:16.415108 5174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:16.419027 5174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:16.419077 5174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:16.419130 5174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:16.423992 5174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab
--principal=oryx/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:40813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:16.424880 5174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:16.425774 5174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:16.435647 5183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:16.435770 5174 server_base.cc:1034] running on GCE node
W20260812 18:59:16.435647 5179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:16.435673 5180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:16.436380 5174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:16.436985 5174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:16.438200 5174 hybrid_clock.cc:648] HybridClock initialized: now 1786561156438180 us; error 31 us; skew 500 ppm
Aug 12 18:59:16 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:16.440878 5174 init.cc:377] Logged in from keytab as oryx/127.29.112.194@KRBTEST.COM (short username oryx)
I20260812 18:59:16.441772 5174 webserver.cc:460] Webserver started at http://127.29.112.194:37535/ using document root <none> and password file <none>
I20260812 18:59:16.442356 5174 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:16.442444 5174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:16.442677 5174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:16.444653 5174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "b5f9235f90144386bc27513990c1cf8f"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "e34292aa7e889763b5e3506fb1df52b1"
server_key_iv: "efcd1d58cd4a23e054f02048c0f73c8b"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.445264 5174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "b5f9235f90144386bc27513990c1cf8f"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "e34292aa7e889763b5e3506fb1df52b1"
server_key_iv: "efcd1d58cd4a23e054f02048c0f73c8b"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.449023 5174 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 18:59:16.451362 5192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:16.452574 5174 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:16.452703 5174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "b5f9235f90144386bc27513990c1cf8f"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "e34292aa7e889763b5e3506fb1df52b1"
server_key_iv: "efcd1d58cd4a23e054f02048c0f73c8b"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.452813 5174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:16.466286 5174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:16.469329 5174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:16.469537 5174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:16.470204 5174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:16.471019 5174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:16.471099 5174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:16.471205 5174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:16.471256 5174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:16.479916 5174 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:39297
I20260812 18:59:16.479967 5323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:39297 every 8 connection(s)
I20260812 18:59:16.480830 5174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:16.482956 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 5174
I20260812 18:59:16.483071 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:16.483405 30147 external_mini_cluster.cc:1428] Setting key c968b88054a2bd499fc97a459bf5789b
Aug 12 18:59:16 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.29.112.195@KRBTEST.COM; defaulting to no policy
I20260812 18:59:16.497176 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.482891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:37933 (local address 127.29.112.254:40813)
0812 18:59:16.483063 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:16.483068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:16.484041 (+ 973us) server_negotiation.cc:407] Connection header received
0812 18:59:16.485847 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:16.485851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:16.485907 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:16.485971 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:16.487553 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.488411 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.490243 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.490449 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.490525 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.493419 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:16.493422 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:16.493424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:16.493447 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:16.495139 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.495650 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.495655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.495657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.495738 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.496096 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.496099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.496100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.496287 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:16.496397 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:16.496871 (+ 474us) server_negotiation.cc:299] Negotiation successful
0812 18:59:16.496994 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.195@KRBTEST.COM" created.
I20260812 18:59:16.498054 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.483344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.194:37933)
0812 18:59:16.483869 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:16.483933 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:16.485584 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:16.486176 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:16.486188 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:16.486714 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:16.487407 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.487422 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.488549 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.488554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.490100 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.490114 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.490601 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.490605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.490819 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.491519 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:16.491545 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:16.493254 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:16.495259 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.495265 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.495278 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.495547 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.495828 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.495832 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.495834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.495989 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.496396 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:16.496402 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:16.496635 (+ 233us) client_negotiation.cc:769] Sending connection context
0812 18:59:16.496818 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 18:59:16.497126 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":125,"threads_started":1}
I20260812 18:59:16.499189 5324 heartbeater.cc:347] Connected to a master server at 127.29.112.254:40813
I20260812 18:59:16.499454 5324 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:16.499912 5324 heartbeater.cc:511] Master 127.29.112.254:40813 requested a full tablet report, sending...
I20260812 18:59:16.500928 4955 ts_manager.cc:194] Registered new tserver with Master: b5f9235f90144386bc27513990c1cf8f (127.29.112.194:39297)
I20260812 18:59:16.501477 4955 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:37933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab.
Entry for principal oryx/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab.
I20260812 18:59:16.513042 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:40813
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab
--principal=oryx/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:16.624948 5330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:16.625201 5330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:16.625244 5330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:16.629232 5330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:16.629289 5330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:16.629346 5330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:16.634565 5330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab
--principal=oryx/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:40813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:16.635510 5330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:16.636461 5330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:16.646713 5336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:16.646713 5335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:16.646714 5338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:16.646878 5330 server_base.cc:1034] running on GCE node
I20260812 18:59:16.647512 5330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:16.648130 5330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:16.649300 5330 hybrid_clock.cc:648] HybridClock initialized: now 1786561156649281 us; error 62 us; skew 500 ppm
Aug 12 18:59:16 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:16.652254 5330 init.cc:377] Logged in from keytab as oryx/127.29.112.195@KRBTEST.COM (short username oryx)
I20260812 18:59:16.653115 5330 webserver.cc:460] Webserver started at http://127.29.112.195:34395/ using document root <none> and password file <none>
I20260812 18:59:16.653676 5330 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:16.653736 5330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:16.654031 5330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:16.656123 5330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "99f725af76044b9db2a594397d1158ee"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "7c651961f2acbf763af81e5417e263e0"
server_key_iv: "ce911ce561db2daa2b25fa8945d6b101"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.656702 5330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "99f725af76044b9db2a594397d1158ee"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "7c651961f2acbf763af81e5417e263e0"
server_key_iv: "ce911ce561db2daa2b25fa8945d6b101"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.660810 5330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 18:59:16.663291 5347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:16.664510 5330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:16.664630 5330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "99f725af76044b9db2a594397d1158ee"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "7c651961f2acbf763af81e5417e263e0"
server_key_iv: "ce911ce561db2daa2b25fa8945d6b101"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.664739 5330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:16.677667 5330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:16.680946 5330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:16.681113 5330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:16.681725 5330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:16.682605 5330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:16.682651 5330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:16.682727 5330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:16.682765 5330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:16.691551 5330 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:39415
I20260812 18:59:16.691642 5479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:39415 every 8 connection(s)
I20260812 18:59:16.692462 5330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:59:16.699313 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 5330
I20260812 18:59:16.699460 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:16.699770 30147 external_mini_cluster.cc:1428] Setting key 564f334bd886955c10d2347e3dc849ca
Aug 12 18:59:16 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:16.705938 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.694346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:50817 (local address 127.29.112.254:40813)
0812 18:59:16.694501 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:16.694505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:16.695288 (+ 783us) server_negotiation.cc:407] Connection header received
0812 18:59:16.696669 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:16.696673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:16.696731 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:16.696834 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:16.697981 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.698519 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.699978 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.700200 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.700333 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.702769 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:16.702772 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:16.702773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:16.702796 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:16.704035 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.704504 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.704507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.704508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.704583 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.705026 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.705028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.705029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.705188 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:16.705266 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:16.705664 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 18:59:16.705752 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 18:59:16.706687 5482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.694686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.195:50817)
0812 18:59:16.695126 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:16.695168 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:16.696456 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:16.696997 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:16.697009 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:16.697351 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:16.697785 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.697797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.698648 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.698658 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.699845 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.699856 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.700367 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.700371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.700564 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.701156 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:16.701172 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:16.702598 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:16.704172 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.704176 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.704186 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.704425 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.704662 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.704664 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.704665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.704787 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.705273 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:16.705277 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:16.705473 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 18:59:16.705617 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 18:59:16.705899 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":114,"threads_started":1}
I20260812 18:59:16.707733 5480 heartbeater.cc:347] Connected to a master server at 127.29.112.254:40813
I20260812 18:59:16.707950 5480 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:16.708410 5480 heartbeater.cc:511] Master 127.29.112.254:40813 requested a full tablet report, sending...
I20260812 18:59:16.709487 4955 ts_manager.cc:194] Registered new tserver with Master: 99f725af76044b9db2a594397d1158ee (127.29.112.195:39415)
I20260812 18:59:16.710167 4955 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:50817
I20260812 18:59:16.713548 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.112.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.253.keytab.
Entry for principal oryx/127.29.112.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.253.keytab.
I20260812 18:59:16.741252 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.253:35525
--webserver_interface=127.29.112.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.253.keytab
--principal=oryx/127.29.112.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:40813,127.29.112.253:35525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:16.850064 5485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:16.850291 5485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:16.850327 5485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:16.854070 5485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:16.854115 5485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:16.854133 5485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:16.854149 5485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:16.854163 5485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:16.859244 5485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:40813,127.29.112.253:35525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.253.keytab
--principal=oryx/127.29.112.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.253:35525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.112.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
W20260812 18:59:16.859395 5485 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.112.254:40813,127.29.112.253:35525'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 18:59:16.860149 5485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:16.861012 5485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:16.869251 5491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:16.869375 5485 server_base.cc:1034] running on GCE node
W20260812 18:59:16.869234 5493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:16.869251 5490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:16.870159 5485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:16.871100 5485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:16.872296 5485 hybrid_clock.cc:648] HybridClock initialized: now 1786561156872268 us; error 43 us; skew 500 ppm
Aug 12 18:59:16 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:16.874975 5485 init.cc:377] Logged in from keytab as oryx/127.29.112.253@KRBTEST.COM (short username oryx)
I20260812 18:59:16.875821 5485 webserver.cc:460] Webserver started at http://127.29.112.253:46035/ using document root <none> and password file <none>
I20260812 18:59:16.876374 5485 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:16.876479 5485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:16.876704 5485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:16.878690 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data/instance:
uuid: "1308435398344058b8a7435586e31d70"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "167a2dda16a3bdf8a68efa1d02b17545"
server_key_iv: "44460fc59abe4f587a18c999d93a194d"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.879256 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/wal/instance:
uuid: "1308435398344058b8a7435586e31d70"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "167a2dda16a3bdf8a68efa1d02b17545"
server_key_iv: "44460fc59abe4f587a18c999d93a194d"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.882889 5485 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 18:59:16.885265 5500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:16.886436 5485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:16.886580 5485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/wal
uuid: "1308435398344058b8a7435586e31d70"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "167a2dda16a3bdf8a68efa1d02b17545"
server_key_iv: "44460fc59abe4f587a18c999d93a194d"
server_key_version: "encryptionkey@0"
I20260812 18:59:16.886684 5485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:16.912869 5485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:16.916062 5485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:16.916270 5485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:16.924839 5485 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.253:35525
I20260812 18:59:16.925096 5559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.253:35525 every 8 connection(s)
I20260812 18:59:16.925941 5485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
I20260812 18:59:16.927636 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 5485
I20260812 18:59:16.928918 5560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:16.933125 5560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 18:59:16 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.253@KRBTEST.COM
Aug 12 18:59:16 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.253@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:16.943665 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.929345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47588 (local address 127.29.112.253:35525)
0812 18:59:16.929843 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:16.929864 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:16.929916 (+ 52us) server_negotiation.cc:407] Connection header received
0812 18:59:16.930602 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:16.930619 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:16.930963 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:16.931240 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:16.932207 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.933275 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.934376 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.935134 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.935297 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.938329 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:16.938335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:16.938348 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:16.938380 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:16.940281 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.940744 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.940749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.940754 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.940878 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.941508 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.941511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.941513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.942091 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:16.942299 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:16.942517 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 18:59:16.942792 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":190,"threads_started":1}
I20260812 18:59:16.945187 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.934946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43364 (local address 127.29.112.254:40813)
0812 18:59:16.935076 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:16.935080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:16.935515 (+ 435us) server_negotiation.cc:407] Connection header received
0812 18:59:16.935656 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:16.935660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:16.935714 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:16.935808 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:16.936466 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.936995 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.938315 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.938563 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.938640 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.941029 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:16.941032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:16.941034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:16.941057 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:16.942644 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.943078 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.943081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.943083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.943166 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.943594 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.943597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.943598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.944558 (+ 960us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:16.944685 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:16.944995 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 18:59:16.945083 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 18:59:16.945283 5564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.934928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43364)
0812 18:59:16.935366 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:16.935415 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:16.935546 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:16.935851 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:16.935856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:16.936227 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:16.936341 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.936347 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.937180 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.937183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.938093 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.938103 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.938672 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.938679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.938782 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.939227 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:16.939245 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:16.940901 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:16.942756 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.942762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.942766 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.942972 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.943387 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.943390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.943392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.943494 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.944697 (+ 1203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:16.944703 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:16.944806 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 18:59:16.944945 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 18:59:16.945114 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":189,"threads_started":1}
I20260812 18:59:16.946530 5560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 18:59:16 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.253@KRBTEST.COM for oryx/127.29.112.253@KRBTEST.COM
I20260812 18:59:16.954293 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.947033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47604 (local address 127.29.112.253:35525)
0812 18:59:16.947134 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:16.947137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:16.947150 (+ 13us) server_negotiation.cc:407] Connection header received
0812 18:59:16.947247 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:16.947252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:16.947284 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:16.947359 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:16.947690 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.948466 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.949180 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.949428 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.949603 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.951488 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:16.951491 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:16.951494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:16.951524 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:16.952757 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.953164 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.953168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.953170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.953239 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.953475 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.953481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.953482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.953657 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:16.953756 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:16.954025 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 18:59:16.954161 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":33}
I20260812 18:59:16.954293 5564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.946941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:35525 (local address 127.0.0.1:47604)
0812 18:59:16.947070 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:16.947085 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:16.947157 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:16.947370 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:16.947373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:16.947518 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:16.947587 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.947592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.948579 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.948582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.949062 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.949068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.949540 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.949543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.949695 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.950088 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:16.950097 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:16.951366 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:16.952890 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.952895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.952898 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.953079 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.953321 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.953325 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.953327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.953400 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.953756 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:16.953759 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:16.953862 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 18:59:16.953968 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 18:59:16.954113 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":19}
I20260812 18:59:16.957394 5560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: Bootstrap starting.
I20260812 18:59:16.959661 5560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:16.960351 5560 log.cc:824] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:16.962306 5560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: No bootstrap required, opened a new log
I20260812 18:59:16.965034 5560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } }
I20260812 18:59:16.965226 5560 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:16.965284 5560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1308435398344058b8a7435586e31d70, State: Initialized, Role: FOLLOWER
I20260812 18:59:16.965698 5560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } }
I20260812 18:59:16.966197 5566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } } }
I20260812 18:59:16.966341 5566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 18:59:16.966846 5560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 18:59:16.972276 5579 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 18:59:16.972368 5579 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 18:59:16.973031 5485 master_runner.cc:428] Detected that this master 1308435398344058b8a7435586e31d70 is joining an existing cluster
I20260812 18:59:16.973104 5485 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.112.253:35525
I20260812 18:59:16.983382 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.975351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43380 (local address 127.29.112.254:40813)
0812 18:59:16.975515 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:16.975520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:16.975857 (+ 337us) server_negotiation.cc:407] Connection header received
0812 18:59:16.976010 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:16.976015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:16.976076 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:16.976163 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:16.976606 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.977162 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.978090 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.978463 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.978543 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.979988 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:16.979991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:16.979994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:16.980020 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:16.981510 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.981997 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.982000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.982002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.982107 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.982465 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.982468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.982470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.982726 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:16.982828 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:16.983097 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 18:59:16.983230 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 18:59:16.983593 5587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.975517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:35525 (local address 127.0.0.1:47620)
0812 18:59:16.975788 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:16.975802 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:16.975914 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:16.976242 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:16.976245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:16.976519 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:16.976649 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.976657 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.977620 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.977623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.978562 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.978583 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.979167 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.979170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.979251 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.979681 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:16.979696 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:16.980135 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:16.981911 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.981914 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.981916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.982161 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.982462 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.982464 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.982466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.982581 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.983006 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:16.983009 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:16.983097 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 18:59:16.983206 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 18:59:16.983361 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":24,"thread_start_us":89,"threads_started":1}
I20260812 18:59:16.983982 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.975687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47620 (local address 127.29.112.253:35525)
0812 18:59:16.975788 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:16.975793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:16.975862 (+ 69us) server_negotiation.cc:407] Connection header received
0812 18:59:16.976040 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:16.976043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:16.976084 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:16.976148 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:16.976769 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.977488 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.978700 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.978982 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.979045 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.980263 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:16.980266 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:16.980269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:16.980289 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:16.981775 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.982248 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.982251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.982252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.982354 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.982677 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.982682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.982684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.982899 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:16.983018 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:16.983272 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 18:59:16.983429 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":41,"spinlock_wait_cycles":31872}
I20260812 18:59:16.984151 5588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.975260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43380)
0812 18:59:16.975779 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:16.975794 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:16.975891 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:16.976178 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:16.976180 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:16.976390 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:16.976492 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.976498 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.977279 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.977284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.977956 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.977969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.978742 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.978748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.978852 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.979358 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:16.979370 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:16.979853 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:16.981618 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.981623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.981624 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.981887 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.982227 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.982230 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.982233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.982334 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.982823 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:16.982826 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:16.982900 (+ 74us) client_negotiation.cc:769] Sending connection context
0812 18:59:16.983829 (+ 929us) client_negotiation.cc:240] Negotiation successful
0812 18:59:16.983992 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":35,"spinlock_wait_cycles":3200,"thread_start_us":209,"threads_started":1}
Aug 12 18:59:16 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.254@KRBTEST.COM for oryx/127.29.112.253@KRBTEST.COM
I20260812 18:59:16.996847 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.988219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47622 (local address 127.29.112.253:35525)
0812 18:59:16.988320 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:16.988324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:16.988593 (+ 269us) server_negotiation.cc:407] Connection header received
0812 18:59:16.988748 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:16.988751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:16.988794 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:16.988869 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:16.989519 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.990282 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.991161 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.991452 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.991545 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.993644 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:16.993647 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:16.993650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:16.993673 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:16.995131 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.995544 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.995547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.995549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.995628 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:16.995869 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:16.995872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:16.995874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:16.996112 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:16.996213 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:16.996606 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 18:59:16.996710 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 18:59:16.996927 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:16.988165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:35525 (local address 127.0.0.1:47622)
0812 18:59:16.988478 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:16.988520 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:16.988636 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:16.988904 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:16.988908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:16.989234 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:16.989398 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.989406 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.990377 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.990379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.991040 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:16.991049 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:16.991556 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:16.991558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:16.991640 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:16.992206 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:16.992218 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:16.993523 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:16.995227 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.995235 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.995239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.995448 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.995713 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:16.995715 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:16.995716 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:16.995787 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:16.996233 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:16.996242 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:16.996359 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 18:59:16.996536 (+ 177us) client_negotiation.cc:240] Negotiation successful
0812 18:59:16.996759 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":79,"threads_started":1}
I20260812 18:59:16.998332 4955 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.112.253:35525: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1308435398344058b8a7435586e31d70"
member_type: NON_VOTER
last_known_addr {
host: "127.29.112.253"
port: 35525
}
attrs {
promote: true
}
}
dest_uuid: "d5d24a92401844d588331250ab11bd13"
cas_config_opid_index: -1
I20260812 18:59:16.999006 4955 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } }
I20260812 18:59:17.001034 5591 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1308435398344058b8a7435586e31d70 (127.29.112.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } } }
I20260812 18:59:17.001559 5531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:17.002943 5531 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Refusing update from remote peer d5d24a92401844d588331250ab11bd13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 18:59:17.003176 5591 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.112.253:35525
I20260812 18:59:17.003358 5591 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } } }
I20260812 18:59:17.003458 5591 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:17.003693 5591 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:17.003929 5485 master.cc:537] Master@127.29.112.253:35525 shutting down...
W20260812 18:59:17.005709 4934 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 18:59:17.006752 4934 consensus_peers.cc:487] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 -> Peer 1308435398344058b8a7435586e31d70 (127.29.112.253:35525): Couldn't send request to peer 1308435398344058b8a7435586e31d70. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 18:59:17.008385 5485 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:17.008698 5485 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:17.008805 5485 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: stopping tablet replica
WARNING: no policy specified for oryx/127.29.112.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.252@KRBTEST.COM" created.
I20260812 18:59:17.014684 5485 master.cc:552] Master@127.29.112.253:35525 shutdown complete.
I20260812 18:59:17.014770 5485 master_runner.cc:305] Clearing existing system tablet
I20260812 18:59:17.015081 5485 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:17.015183 5485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:17.016381 5485 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:17.017278 5599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.017480 5485 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:17.017617 5485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:17.017656 5485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/wal
uuid: "1308435398344058b8a7435586e31d70"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "167a2dda16a3bdf8a68efa1d02b17545"
server_key_iv: "44460fc59abe4f587a18c999d93a194d"
server_key_version: "encryptionkey@0"
I20260812 18:59:17.019570 5485 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:17.021763 5485 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 18:59:17.021934 5485 log.cc:1196] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 18:59:17.022974 5485 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: Deleting consensus metadata
I20260812 18:59:17.023180 5485 master_runner.cc:315] Copying system tablet from 127.29.112.254:40813
I20260812 18:59:17.024506 5485 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: tablet copy: Beginning tablet copy session from remote peer at address 127.29.112.254:40813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.252.keytab.
Entry for principal oryx/127.29.112.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.252.keytab.
I20260812 18:59:17.029700 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.252:45951
--webserver_interface=127.29.112.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.252.keytab
--principal=oryx/127.29.112.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:40813,127.29.112.253:35525,127.29.112.252:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 18:59:17.033227 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.025264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.29.112.254:40813)
0812 18:59:17.025385 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.025390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.025460 (+ 70us) server_negotiation.cc:407] Connection header received
0812 18:59:17.025622 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.025628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.025683 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.025760 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.026280 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.026879 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.028013 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.028227 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.028370 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.029521 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.029524 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.029527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.029551 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.031287 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.031809 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.031814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.031817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.031949 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.032272 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.032278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.032281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.032563 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.032682 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.033013 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.033113 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 18:59:17.033270 5606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.025097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43386)
0812 18:59:17.025385 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.025400 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.025521 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.025781 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.025784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.026034 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.026147 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.026155 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.026980 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.026983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.027872 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.027881 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.028361 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.028365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.028455 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.028960 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.028974 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.029366 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.031417 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.031419 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.031422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.031696 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.032067 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.032070 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.032072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.032165 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.032693 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.032696 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.032814 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.032952 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.033095 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":130,"threads_started":1}
I20260812 18:59:17.034142 4976 tablet_copy_service.cc:140] P d5d24a92401844d588331250ab11bd13: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1308435398344058b8a7435586e31d70 ({username='oryx', principal='oryx/127.29.112.253@KRBTEST.COM'} at 127.0.0.1:43386)
I20260812 18:59:17.034322 4976 tablet_copy_service.cc:161] P d5d24a92401844d588331250ab11bd13: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1308435398344058b8a7435586e31d70 at {username='oryx', principal='oryx/127.29.112.253@KRBTEST.COM'} at 127.0.0.1:43386: session id = 1308435398344058b8a7435586e31d70-00000000000000000000000000000000
I20260812 18:59:17.035880 4976 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 18:59:17.037732 5485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 18:59:17.039975 5485 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: tablet copy: Starting download of 0 data blocks...
I20260812 18:59:17.040205 5485 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: tablet copy: Starting download of 1 WAL segments...
I20260812 18:59:17.042047 5485 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 18:59:17.043808 4976 tablet_copy_service.cc:342] P d5d24a92401844d588331250ab11bd13: Request end of tablet copy session 1308435398344058b8a7435586e31d70-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.112.253@KRBTEST.COM'} at 127.0.0.1:43386
I20260812 18:59:17.043942 4976 tablet_copy_service.cc:433] P d5d24a92401844d588331250ab11bd13: ending tablet copy session 1308435398344058b8a7435586e31d70-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1308435398344058b8a7435586e31d70
I20260812 18:59:17.051494 5485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:17.053191 5610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.053334 5611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.053699 5613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:17.054008 5485 server_base.cc:1034] running on GCE node
I20260812 18:59:17.054184 5485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:17.054560 5485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:17.055698 5485 hybrid_clock.cc:648] HybridClock initialized: now 1786561157055706 us; error 46 us; skew 500 ppm
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:17.057446 5485 init.cc:377] Logged in from keytab as oryx/127.29.112.253@KRBTEST.COM (short username oryx)
I20260812 18:59:17.058035 5485 webserver.cc:460] Webserver started at http://127.29.112.253:37031/ using document root <none> and password file <none>
I20260812 18:59:17.058261 5485 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:17.058324 5485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:17.059576 5485 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:17.060565 5621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.060833 5485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:17.060936 5485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/wal
uuid: "1308435398344058b8a7435586e31d70"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "167a2dda16a3bdf8a68efa1d02b17545"
server_key_iv: "44460fc59abe4f587a18c999d93a194d"
server_key_version: "encryptionkey@0"
I20260812 18:59:17.061048 5485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:17.075963 5485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:17.078218 5485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:17.084566 5485 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.253:35525
I20260812 18:59:17.085178 5485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
I20260812 18:59:17.085942 5679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.253:35525 every 8 connection(s)
I20260812 18:59:17.087141 5680 sys_catalog.cc:263] Verifying existing consensus state
I20260812 18:59:17.088011 5680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: Bootstrap starting.
I20260812 18:59:17.096297 5680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 18:59:17.096625 5680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: Bootstrap complete.
I20260812 18:59:17.097127 5680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } }
I20260812 18:59:17.097286 5680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1308435398344058b8a7435586e31d70, State: Initialized, Role: LEARNER
I20260812 18:59:17.097400 5680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } }
I20260812 18:59:17.097728 5683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } } }
I20260812 18:59:17.097924 5680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:17.097936 5683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: This master's current role is: LEARNER
I20260812 18:59:17.101241 5699 catalog_manager.cc:1259] Loaded cluster ID: 85d01156aff54b33b86e23f078b6ba6e
I20260812 18:59:17.101351 5699 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: loading cluster ID for follower catalog manager: success
I20260812 18:59:17.103060 5699 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: acquiring CA information for follower catalog manager: success
I20260812 18:59:17.104095 5699 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 18:59:17.176755 5607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:17.177078 5607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:17.177150 5607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:17.182394 5607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:17.182451 5607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:17.182471 5607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:17.182511 5607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:17.182541 5607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:17.188033 5607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:40813,127.29.112.253:35525,127.29.112.252:45951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.252.keytab
--principal=oryx/127.29.112.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.252:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.112.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:17.189036 5607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:17.190055 5607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:17.198107 5705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.198127 5704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.198112 5707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:17.198279 5607 server_base.cc:1034] running on GCE node
I20260812 18:59:17.198838 5607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:17.199904 5607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:17.201092 5607 hybrid_clock.cc:648] HybridClock initialized: now 1786561157201090 us; error 56 us; skew 500 ppm
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:17.203804 5607 init.cc:377] Logged in from keytab as oryx/127.29.112.252@KRBTEST.COM (short username oryx)
I20260812 18:59:17.204609 5607 webserver.cc:460] Webserver started at http://127.29.112.252:37837/ using document root <none> and password file <none>
I20260812 18:59:17.205140 5607 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:17.205230 5607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:17.205451 5607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:17.207597 5607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data/instance:
uuid: "93e1da7fa1ee43d0b3c50ec5f003d964"
format_stamp: "Formatted at 2026-08-12 18:59:17 on dist-test-slave-6nmv"
server_key: "5719994d1bc70a3a42fd85f368b2f810"
server_key_iv: "25ec8b6b2b1d9dc08c8fa80feefd1d92"
server_key_version: "encryptionkey@0"
I20260812 18:59:17.208213 5607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/wal/instance:
uuid: "93e1da7fa1ee43d0b3c50ec5f003d964"
format_stamp: "Formatted at 2026-08-12 18:59:17 on dist-test-slave-6nmv"
server_key: "5719994d1bc70a3a42fd85f368b2f810"
server_key_iv: "25ec8b6b2b1d9dc08c8fa80feefd1d92"
server_key_version: "encryptionkey@0"
I20260812 18:59:17.212030 5607 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260812 18:59:17.214435 5716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.215708 5607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 18:59:17.215838 5607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/wal
uuid: "93e1da7fa1ee43d0b3c50ec5f003d964"
format_stamp: "Formatted at 2026-08-12 18:59:17 on dist-test-slave-6nmv"
server_key: "5719994d1bc70a3a42fd85f368b2f810"
server_key_iv: "25ec8b6b2b1d9dc08c8fa80feefd1d92"
server_key_version: "encryptionkey@0"
I20260812 18:59:17.215970 5607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:17.228674 5607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:17.231540 5607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:17.231766 5607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:17.239022 5607 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.252:45951
I20260812 18:59:17.239398 5774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.252:45951 every 8 connection(s)
I20260812 18:59:17.240038 5607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
I20260812 18:59:17.243043 5775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:17.247116 5775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } has no permanent_uuid. Determining permanent_uuid...
I20260812 18:59:17.248190 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 5607
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.252@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.252@KRBTEST.COM
I20260812 18:59:17.261353 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.249087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43398 (local address 127.29.112.254:40813)
0812 18:59:17.249213 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.249218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.250197 (+ 979us) server_negotiation.cc:407] Connection header received
0812 18:59:17.250940 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.250943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.250994 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.251083 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.252509 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.253271 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.254554 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.254869 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.255010 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.257541 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.257546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.257549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.257579 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.258995 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.259524 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.259527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.259529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.259626 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.260034 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.260036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.260037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.260299 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.260400 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.261003 (+ 603us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.261158 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 18:59:17.262108 5779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.249399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43398)
0812 18:59:17.250044 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.250089 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.250720 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.251890 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.251895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.252210 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.252363 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.252372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.253390 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.253394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.254439 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.254452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.255063 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.255068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.255220 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.255678 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.255703 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.257389 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.259099 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.259109 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.259118 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.259398 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.259741 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.259745 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.259747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.259927 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.260405 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.260415 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.260703 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.260925 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.261247 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":300,"threads_started":1}
I20260812 18:59:17.262111 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.249399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.29.112.252:45951)
0812 18:59:17.249961 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.249980 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.250024 (+ 44us) server_negotiation.cc:407] Connection header received
0812 18:59:17.250704 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.250721 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.250960 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.251127 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.251855 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.253080 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.254245 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.254758 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.254904 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.258010 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.258017 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.258030 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.258055 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.259695 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.260166 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.260171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.260173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.260252 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.260557 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.260559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.260560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.260781 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.260991 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.261252 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.261468 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":434,"thread_start_us":280,"threads_started":1}
I20260812 18:59:17.263311 5775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.252@KRBTEST.COM for oryx/127.29.112.253@KRBTEST.COM
I20260812 18:59:17.271757 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.263908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47624 (local address 127.29.112.253:35525)
0812 18:59:17.264445 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.264458 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.264470 (+ 12us) server_negotiation.cc:407] Connection header received
0812 18:59:17.264514 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.264516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.264552 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.264624 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.264987 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.265646 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.266453 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.266687 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.266765 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.268678 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.268681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.268683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.268703 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.270264 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.270664 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.270666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.270667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.270740 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.270999 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.271001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.271002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.271147 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.271215 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.271537 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.271643 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":208,"threads_started":1}
I20260812 18:59:17.271767 5779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.263804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:35525 (local address 127.0.0.1:47624)
0812 18:59:17.263953 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.263966 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.264042 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.264658 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.264660 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.264779 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.264868 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.264872 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.265745 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.265748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.266352 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.266360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.266780 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.266782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.266848 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.267348 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.267361 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.268566 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.270370 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.270372 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.270374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.270574 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.270825 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.270827 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.270828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.270923 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.271223 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.271224 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.271295 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.271473 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.271601 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 18:59:17.272512 5775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.252@KRBTEST.COM for oryx/127.29.112.252@KRBTEST.COM
I20260812 18:59:17.279922 5779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.273006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:45951 (local address 127.0.0.1:42322)
0812 18:59:17.273192 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.273207 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.273296 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.273505 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.273509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.273674 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.273758 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.273765 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.274540 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.274543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.275024 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.275031 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.275464 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.275467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.275547 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.275989 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.276004 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.277190 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.278647 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.278649 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.278650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.278845 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.279117 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.279119 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.279120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.279203 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.279508 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.279510 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.279584 (+ 74us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.279671 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.279777 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260812 18:59:17.279942 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.273074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.29.112.252:45951)
0812 18:59:17.273191 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.273194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.273267 (+ 73us) server_negotiation.cc:407] Connection header received
0812 18:59:17.273374 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.273376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.273407 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.273466 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.273918 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.274420 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.275132 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.275363 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.275468 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.277317 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.277320 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.277322 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.277341 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.278549 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.278931 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.278933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.278934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.279000 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.279281 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.279283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.279284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.279430 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.279502 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.279722 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.279830 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":48}
I20260812 18:59:17.283538 5775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: Bootstrap starting.
I20260812 18:59:17.285753 5775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:17.286525 5775 log.cc:824] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:17.288404 5775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: No bootstrap required, opened a new log
I20260812 18:59:17.291256 5775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } }
I20260812 18:59:17.291451 5775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:17.291513 5775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e1da7fa1ee43d0b3c50ec5f003d964, State: Initialized, Role: FOLLOWER
I20260812 18:59:17.291939 5775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } }
I20260812 18:59:17.292546 5782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } } }
I20260812 18:59:17.292695 5782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 18:59:17.293452 5775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:17.297032 5168 heartbeater.cc:503] Master 127.29.112.254:40813 was elected leader, sending a full tablet report...
W20260812 18:59:17.298944 5795 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 18:59:17.299028 5795 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 18:59:17.302598 5607 master_runner.cc:428] Detected that this master 93e1da7fa1ee43d0b3c50ec5f003d964 is joining an existing cluster
I20260812 18:59:17.302660 5607 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.112.252:45951
I20260812 18:59:17.314271 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.305068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.29.112.254:40813)
0812 18:59:17.305224 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.305231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.305988 (+ 757us) server_negotiation.cc:407] Connection header received
0812 18:59:17.306231 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.306234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.306285 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.306342 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.307147 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.307742 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.308601 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.308827 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.308951 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.310507 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.310516 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.310519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.310551 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.312354 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.312842 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.312846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.312848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.312947 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.313259 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.313265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.313267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.313482 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.313637 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.313979 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.314100 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 18:59:17.314291 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.305342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43408)
0812 18:59:17.305903 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.305916 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.306086 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.306519 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.306522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.306841 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.307008 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.307017 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.307856 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.307869 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.308452 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.308461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.308965 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.308969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.309094 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.309697 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.309710 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.310332 (+ 622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.312495 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.312498 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.312499 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.312711 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.313058 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.313061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.313063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.313158 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.313624 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.313627 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.313764 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.313921 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.314088 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":23,"thread_start_us":104,"threads_started":1}
I20260812 18:59:17.315140 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.305583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47638 (local address 127.29.112.253:35525)
0812 18:59:17.305705 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.305709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.305777 (+ 68us) server_negotiation.cc:407] Connection header received
0812 18:59:17.306073 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.306077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.306131 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.306186 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.307254 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.308039 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.309290 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.309666 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.309754 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.311268 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.311272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.311274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.311312 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.313145 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.313661 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.313665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.313667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.313763 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.314106 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.314109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.314111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.314310 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.314374 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.314826 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.314987 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 18:59:17.315631 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.305053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:35525 (local address 127.0.0.1:47638)
0812 18:59:17.305690 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.305707 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.305908 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.306403 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.306406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.306891 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.307038 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.307045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.308187 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.308190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.309146 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.309156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.309918 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.309921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.310042 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.310572 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.310589 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.311120 (+ 531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.313294 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.313297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.313299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.313551 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.313907 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.313909 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.313911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.314005 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.314445 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.314447 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.314621 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.315329 (+ 708us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.315466 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":32,"thread_start_us":163,"threads_started":1}
I20260812 18:59:17.315971 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 5024
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.254@KRBTEST.COM for oryx/127.29.112.252@KRBTEST.COM
I20260812 18:59:17.343238 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:41303
--local_ip_for_outbound_sockets=127.29.112.193
--tserver_master_addrs=127.29.112.254:40813
--webserver_port=37477
--webserver_interface=127.29.112.193
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab
--principal=oryx/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.112.254:40813,127.29.112.253:35525
--tserver_master_addrs=127.29.112.254:40813,127.29.112.253:35525,127.29.112.252:45951 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 18:59:17.346359 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.305553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.29.112.252:45951)
0812 18:59:17.305664 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.305669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.305881 (+ 212us) server_negotiation.cc:407] Connection header received
0812 18:59:17.306050 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.306054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.306093 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.306146 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.335023 (+ 28877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.335827 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.337126 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.337552 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.337645 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.342454 (+ 4809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.342458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.342460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.342481 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.344527 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.345052 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.345055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.345057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.345125 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.345366 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.345369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.345371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.345624 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.345749 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.346070 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.346184 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":46}
I20260812 18:59:17.346751 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.334537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:45951 (local address 127.0.0.1:42344)
0812 18:59:17.334738 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.334766 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.334851 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.336732 (+ 1881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.336737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.336991 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.337089 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.337096 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.337997 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.338002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.338780 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.338791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.339418 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.339422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.339526 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.340031 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.340053 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.341392 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.344132 (+ 2740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.344138 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.344144 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.344473 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.344945 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.344952 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.344957 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.345117 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.346082 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.346088 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.346214 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.346317 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.346519 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260812 18:59:17.347265 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.305105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:45951 (local address 127.0.0.1:42338)
0812 18:59:17.305785 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.305799 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.305923 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.306403 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.306406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.331902 (+ 25496us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.332060 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.332073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.335961 (+ 3888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.335967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.336998 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.337012 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.337805 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.337851 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.337978 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.341881 (+ 3903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.341905 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.342320 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.344637 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.344639 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.344642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.344957 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.345213 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.345216 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.345218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.345286 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.345741 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.345744 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.345896 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.346761 (+ 865us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.347093 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":3,"spinlock_wait_cycles":3456,"thread_start_us":130,"threads_started":1}
I20260812 18:59:17.347472 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.334798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.29.112.252:45951)
0812 18:59:17.336453 (+ 1655us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.336462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.336476 (+ 14us) server_negotiation.cc:407] Connection header received
0812 18:59:17.336537 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.336539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.336593 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.336757 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.337216 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.337751 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.338919 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.339279 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.339409 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.341582 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.341586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.341589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.341635 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.343983 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.344623 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.344629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.344634 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.344771 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.345273 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.345279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.345283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.345937 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.346964 (+ 1027us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.347053 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.347196 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":1569,"thread_start_us":104,"threads_started":1}
I20260812 18:59:17.348507 4955 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.112.252:45951: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964"
member_type: NON_VOTER
last_known_addr {
host: "127.29.112.252"
port: 45951
}
attrs {
promote: true
}
}
dest_uuid: "d5d24a92401844d588331250ab11bd13"
cas_config_opid_index: 5
I20260812 18:59:17.349165 4955 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } }
I20260812 18:59:17.350476 5591 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 93e1da7fa1ee43d0b3c50ec5f003d964 (127.29.112.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.352339 5591 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.112.252:45951
I20260812 18:59:17.352586 5591 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.352737 5591 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:17.353435 5607 master.cc:537] Master@127.29.112.252:45951 shutting down...
W20260812 18:59:17.355931 4937 consensus_peers.cc:487] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 -> Peer 93e1da7fa1ee43d0b3c50ec5f003d964 (127.29.112.252:45951): Couldn't send request to peer 93e1da7fa1ee43d0b3c50ec5f003d964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 18:59:17.356757 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.350676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:35525 (local address 127.0.0.1:47648)
0812 18:59:17.350963 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.350977 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.351048 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.351432 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.351435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.351452 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:17.351546 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.351551 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.352644 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.352647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.353926 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.353937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.356314 (+ 2377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.356318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.356416 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.356466 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.356546 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.356618 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260812 18:59:17.356859 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.350971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47648 (local address 127.29.112.253:35525)
0812 18:59:17.351125 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.351130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.351147 (+ 17us) server_negotiation.cc:407] Connection header received
0812 18:59:17.351206 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.351209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.351276 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.351407 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:17.351698 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.352509 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.355186 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.355884 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.356022 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.356162 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.356595 (+ 433us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.356699 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 18:59:17.357453 5649 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 LEARNER]: Refusing update from remote peer d5d24a92401844d588331250ab11bd13: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 18:59:17.358014 5592 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 18:59:17.359756 5607 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:17.360157 5607 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:17.360365 5607 tablet_replica.cc:330] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: stopping tablet replica
I20260812 18:59:17.360239 5649 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 93e1da7fa1ee43d0b3c50ec5f003d964 (127.29.112.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.362730 5683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5d24a92401844d588331250ab11bd13. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.362896 5683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: This master's current role is: LEARNER
I20260812 18:59:17.363019 5683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: NON_VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: true } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.363114 5683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: This master's current role is: LEARNER
I20260812 18:59:17.363369 5592 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13: attempting to promote NON_VOTER 1308435398344058b8a7435586e31d70 to VOTER
I20260812 18:59:17.363919 5592 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } }
I20260812 18:59:17.365372 5649 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 LEARNER]: Refusing update from remote peer d5d24a92401844d588331250ab11bd13: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 18:59:17.365746 5808 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260812 18:59:17.366403 4937 consensus_peers.cc:487] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 -> Peer 93e1da7fa1ee43d0b3c50ec5f003d964 (127.29.112.252:45951): Couldn't send request to peer 93e1da7fa1ee43d0b3c50ec5f003d964. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 18:59:17.367254 5592 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1308435398344058b8a7435586e31d70 (127.29.112.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.367861 5649 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1308435398344058b8a7435586e31d70 (127.29.112.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.369197 5592 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.369710 5592 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:17.369515 5591 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.369877 5591 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:17.372787 5607 master.cc:552] Master@127.29.112.252:45951 shutdown complete.
I20260812 18:59:17.372853 5607 master_runner.cc:305] Clearing existing system tablet
I20260812 18:59:17.373100 5607 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:17.373157 5607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:17.373955 5807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.374168 5807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 18:59:17.374336 5607 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 18:59:17.375203 5812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.375394 5607 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:17.375520 5607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:17.375555 5607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/wal
uuid: "93e1da7fa1ee43d0b3c50ec5f003d964"
format_stamp: "Formatted at 2026-08-12 18:59:17 on dist-test-slave-6nmv"
server_key: "5719994d1bc70a3a42fd85f368b2f810"
server_key_iv: "25ec8b6b2b1d9dc08c8fa80feefd1d92"
server_key_version: "encryptionkey@0"
I20260812 18:59:17.377744 5607 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:17.380390 5607 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 18:59:17.380484 5607 log.cc:1196] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 18:59:17.380731 5607 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: Deleting consensus metadata
I20260812 18:59:17.380908 5607 master_runner.cc:315] Copying system tablet from 127.29.112.254:40813
I20260812 18:59:17.382062 5607 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: tablet copy: Beginning tablet copy session from remote peer at address 127.29.112.254:40813
I20260812 18:59:17.391198 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.382558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43418)
0812 18:59:17.382809 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.382821 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.382932 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.383210 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.383212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.383409 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.383544 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.383551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.384575 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.384578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.385562 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.385573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.386259 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.386263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.386377 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.387012 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.387026 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.387454 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.389480 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.389483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.389485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.389721 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.390131 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.390133 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.390135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.390225 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.390653 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.390656 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.390748 (+ 92us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.390849 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.391014 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":75,"threads_started":1}
I20260812 18:59:17.391242 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.382691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.29.112.254:40813)
0812 18:59:17.382817 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.382822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.382880 (+ 58us) server_negotiation.cc:407] Connection header received
0812 18:59:17.383030 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.383033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.383083 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.383175 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.383686 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.384426 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.385687 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.386134 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.386258 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.387611 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.387616 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.387619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.387649 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.389366 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.389929 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.389932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.389934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.390031 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.390320 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.390323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.390325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.390549 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.390657 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.390932 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.391075 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 18:59:17.392098 4976 tablet_copy_service.cc:140] P d5d24a92401844d588331250ab11bd13: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 93e1da7fa1ee43d0b3c50ec5f003d964 ({username='oryx', principal='oryx/127.29.112.252@KRBTEST.COM'} at 127.0.0.1:43418)
I20260812 18:59:17.392212 4976 tablet_copy_service.cc:161] P d5d24a92401844d588331250ab11bd13: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 93e1da7fa1ee43d0b3c50ec5f003d964 at {username='oryx', principal='oryx/127.29.112.252@KRBTEST.COM'} at 127.0.0.1:43418: session id = 93e1da7fa1ee43d0b3c50ec5f003d964-00000000000000000000000000000000
I20260812 18:59:17.393468 4976 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 18:59:17.395000 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 18:59:17.397748 5607 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: tablet copy: Starting download of 0 data blocks...
I20260812 18:59:17.398005 5607 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: tablet copy: Starting download of 1 WAL segments...
I20260812 18:59:17.399586 5607 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 18:59:17.401345 4976 tablet_copy_service.cc:342] P d5d24a92401844d588331250ab11bd13: Request end of tablet copy session 93e1da7fa1ee43d0b3c50ec5f003d964-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.112.252@KRBTEST.COM'} at 127.0.0.1:43418
I20260812 18:59:17.401427 4976 tablet_copy_service.cc:433] P d5d24a92401844d588331250ab11bd13: ending tablet copy session 93e1da7fa1ee43d0b3c50ec5f003d964-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 93e1da7fa1ee43d0b3c50ec5f003d964
I20260812 18:59:17.408569 5607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:17.410419 5823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.410454 5820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.410748 5821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:17.414033 5607 server_base.cc:1034] running on GCE node
I20260812 18:59:17.414194 5607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:17.414476 5607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:17.415617 5607 hybrid_clock.cc:648] HybridClock initialized: now 1786561157415600 us; error 44 us; skew 500 ppm
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:17.417476 5607 init.cc:377] Logged in from keytab as oryx/127.29.112.252@KRBTEST.COM (short username oryx)
I20260812 18:59:17.418037 5607 webserver.cc:460] Webserver started at http://127.29.112.252:41735/ using document root <none> and password file <none>
I20260812 18:59:17.418241 5607 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:17.418288 5607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:17.419725 5607 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:17.420800 5831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.421038 5607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:17.421142 5607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/wal
uuid: "93e1da7fa1ee43d0b3c50ec5f003d964"
format_stamp: "Formatted at 2026-08-12 18:59:17 on dist-test-slave-6nmv"
server_key: "5719994d1bc70a3a42fd85f368b2f810"
server_key_iv: "25ec8b6b2b1d9dc08c8fa80feefd1d92"
server_key_version: "encryptionkey@0"
I20260812 18:59:17.421298 5607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:17.432480 5607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:17.434926 5607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:17.440243 5607 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.252:45951
I20260812 18:59:17.440282 5890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.252:45951 every 8 connection(s)
I20260812 18:59:17.440976 5607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
I20260812 18:59:17.443154 5891 sys_catalog.cc:263] Verifying existing consensus state
I20260812 18:59:17.444340 5891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: Bootstrap starting.
I20260812 18:59:17.456960 5891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 18:59:17.457341 5891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: Bootstrap complete.
I20260812 18:59:17.458105 5891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } }
I20260812 18:59:17.458280 5891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 93e1da7fa1ee43d0b3c50ec5f003d964, State: Initialized, Role: LEARNER
I20260812 18:59:17.458424 5891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } }
I20260812 18:59:17.458846 5894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.458983 5891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:17.459054 5894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: This master's current role is: LEARNER
I20260812 18:59:17.462986 5906 catalog_manager.cc:1259] Loaded cluster ID: 85d01156aff54b33b86e23f078b6ba6e
I20260812 18:59:17.463048 5906 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: loading cluster ID for follower catalog manager: success
I20260812 18:59:17.465102 5906 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: acquiring CA information for follower catalog manager: success
I20260812 18:59:17.466555 5906 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 18:59:17.495020 5805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:17.495288 5805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:17.495383 5805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:17.499650 5805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:17.499713 5805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:17.499830 5805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:17.503402 5324 heartbeater.cc:503] Master 127.29.112.254:40813 was elected leader, sending a full tablet report...
I20260812 18:59:17.505273 5805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab
--principal=oryx/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:41303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=37477
--tserver_master_addrs=127.29.112.254:40813,127.29.112.253:35525,127.29.112.252:45951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:17.506429 5805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:17.507436 5805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:17.517125 5911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.517204 5912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.517313 5914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:17.517789 5805 server_base.cc:1034] running on GCE node
I20260812 18:59:17.518244 5805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:17.518818 5805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:17.519969 5805 hybrid_clock.cc:648] HybridClock initialized: now 1786561157519939 us; error 43 us; skew 500 ppm
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:17.522899 5805 init.cc:377] Logged in from keytab as oryx/127.29.112.193@KRBTEST.COM (short username oryx)
I20260812 18:59:17.523693 5805 webserver.cc:460] Webserver started at http://127.29.112.193:37477/ using document root <none> and password file <none>
I20260812 18:59:17.524228 5805 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:17.524340 5805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:17.527941 5805 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:17.529784 5923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.530916 5805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:17.531035 5805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "17baa51a90904d65b6a9a07958887c47"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "3ed7ee697ccee0870e5aa493e6fa0b0e"
server_key_iv: "1a1fbac72f1ba25358acd16bb0a21565"
server_key_version: "encryptionkey@0"
I20260812 18:59:17.531531 5805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:17.558506 5805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:17.561766 5805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:17.562006 5805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:17.562641 5805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:17.563562 5805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:17.563639 5805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.563709 5805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:17.563766 5805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.575601 5805 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:41303
I20260812 18:59:17.575731 6047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:41303 every 8 connection(s)
I20260812 18:59:17.576684 5805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:17.583706 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 5805
I20260812 18:59:17.583894 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 5174
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.252@KRBTEST.COM
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.253@KRBTEST.COM
I20260812 18:59:17.595528 6052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.578681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:44253 (local address 127.29.112.252:45951)
0812 18:59:17.579500 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.579506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.579991 (+ 485us) server_negotiation.cc:407] Connection header received
0812 18:59:17.581313 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.581316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.581394 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.581478 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.582728 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.583516 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.585346 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.585769 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.585922 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.590251 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.590257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.590260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.590292 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.593198 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.593758 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.593762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.593763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.593878 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.594230 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.594234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.594235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.594469 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.594548 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.595233 (+ 685us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.595355 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":85,"threads_started":1}
I20260812 18:59:17.595525 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.578972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:38651 (local address 127.29.112.254:40813)
0812 18:59:17.579114 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.579122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.580713 (+ 1591us) server_negotiation.cc:407] Connection header received
0812 18:59:17.581619 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.581622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.581677 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.581761 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.582736 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.583407 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.585980 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.586341 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.586427 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.590309 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.590313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.590316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.590347 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.592496 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.593172 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.593176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.593177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.593270 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.594232 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.594235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.594237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.594469 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.594554 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.595233 (+ 679us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.595355 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 18:59:17.596766 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.579284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:36567 (local address 127.29.112.253:35525)
0812 18:59:17.579409 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.579412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.580140 (+ 728us) server_negotiation.cc:407] Connection header received
0812 18:59:17.581211 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.581215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.581287 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.581372 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.582667 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.583408 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.586955 (+ 3547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.587399 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.587547 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.592966 (+ 5419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.592973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.592979 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.593024 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.594773 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.595264 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.595270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.595274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.595358 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.595652 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.595658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.595661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.595899 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.595978 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.596401 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.596535 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 18:59:17.597334 6053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.579037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:45951 (local address 127.29.112.193:44253)
0812 18:59:17.579856 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.579903 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.580990 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.581565 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.581571 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.582113 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.582579 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.582586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.583673 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.583676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.585193 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.585205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.586147 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.586151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.586349 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.587191 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.587217 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.590105 (+ 2888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.593320 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.593324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.593327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.593647 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.594008 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.594013 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.594015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.594114 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.594668 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.594671 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.594935 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.596025 (+ 1090us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.596304 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":280,"spinlock_wait_cycles":47360,"thread_start_us":438,"threads_started":1}
I20260812 18:59:17.597488 6054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.579125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:35525 (local address 127.29.112.193:36567)
0812 18:59:17.580063 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.580075 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.580990 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.581567 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.581573 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.582087 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.582512 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.582524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.583564 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.583570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.585096 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.585109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.589181 (+ 4072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.589190 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.589333 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.589936 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.589958 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.592766 (+ 2808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.594888 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.594894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.594898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.595156 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.595462 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.595468 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.595472 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.595558 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.596090 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.596096 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.596210 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.596959 (+ 749us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.597140 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"mutex_wait_us":53,"spinlock_wait_cycles":7936,"thread_start_us":465,"threads_started":1}
I20260812 18:59:17.597330 6055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.579043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.193:38651)
0812 18:59:17.580581 (+ 1538us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.580595 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.581005 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.581834 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.581839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.582125 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.582577 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.582586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.583830 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.583833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.585805 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.585839 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.586606 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.586613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.586718 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.587313 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.587330 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.589733 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.592646 (+ 2913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.592654 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.592671 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.593040 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.593666 (+ 626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.593671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.593676 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.594124 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.594621 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.594630 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.594935 (+ 305us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.595808 (+ 873us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.596247 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1457,"mutex_wait_us":1403,"spinlock_wait_cycles":49408,"thread_start_us":431,"threads_started":1}
I20260812 18:59:17.598737 6049 heartbeater.cc:347] Connected to a master server at 127.29.112.252:45951
I20260812 18:59:17.598897 6050 heartbeater.cc:347] Connected to a master server at 127.29.112.253:35525
I20260812 18:59:17.598995 6049 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:17.598995 6050 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:17.599485 6048 heartbeater.cc:347] Connected to a master server at 127.29.112.254:40813
I20260812 18:59:17.599563 6048 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:17.599637 6049 heartbeater.cc:511] Master 127.29.112.252:45951 requested a full tablet report, sending...
I20260812 18:59:17.599726 6048 heartbeater.cc:511] Master 127.29.112.254:40813 requested a full tablet report, sending...
I20260812 18:59:17.600451 6050 heartbeater.cc:511] Master 127.29.112.253:35525 requested a full tablet report, sending...
I20260812 18:59:17.600967 4955 ts_manager.cc:194] Re-registered known tserver with Master: 17baa51a90904d65b6a9a07958887c47 (127.29.112.193:41303)
I20260812 18:59:17.601524 4955 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:38651
I20260812 18:59:17.601531 5639 ts_manager.cc:194] Registered new tserver with Master: 17baa51a90904d65b6a9a07958887c47 (127.29.112.193:41303)
I20260812 18:59:17.601531 5852 ts_manager.cc:194] Registered new tserver with Master: 17baa51a90904d65b6a9a07958887c47 (127.29.112.193:41303)
I20260812 18:59:17.603338 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:39297
--local_ip_for_outbound_sockets=127.29.112.194
--tserver_master_addrs=127.29.112.254:40813
--webserver_port=37535
--webserver_interface=127.29.112.194
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab
--principal=oryx/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.112.254:40813,127.29.112.253:35525
--tserver_master_addrs=127.29.112.254:40813,127.29.112.253:35525,127.29.112.252:45951 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:17.710937 6056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:17.711190 6056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:17.711268 6056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 18:59:17.712033 5480 heartbeater.cc:503] Master 127.29.112.254:40813 was elected leader, sending a full tablet report...
W20260812 18:59:17.715360 6056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:17.715441 6056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:17.715539 6056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:17.720542 6056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab
--principal=oryx/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:39297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=37535
--tserver_master_addrs=127.29.112.254:40813,127.29.112.253:35525,127.29.112.252:45951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:17.721386 6056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:17.722275 6056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:17.731112 6063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.731170 6065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.731302 6062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:17.731467 6056 server_base.cc:1034] running on GCE node
I20260812 18:59:17.731875 6056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:17.732460 6056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:17.733680 6056 hybrid_clock.cc:648] HybridClock initialized: now 1786561157733668 us; error 29 us; skew 500 ppm
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:17.736579 6056 init.cc:377] Logged in from keytab as oryx/127.29.112.194@KRBTEST.COM (short username oryx)
I20260812 18:59:17.737478 6056 webserver.cc:460] Webserver started at http://127.29.112.194:37535/ using document root <none> and password file <none>
I20260812 18:59:17.738112 6056 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:17.738246 6056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:17.742208 6056 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 18:59:17.744195 6072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.745361 6056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:17.745497 6056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "b5f9235f90144386bc27513990c1cf8f"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "e34292aa7e889763b5e3506fb1df52b1"
server_key_iv: "efcd1d58cd4a23e054f02048c0f73c8b"
server_key_version: "encryptionkey@0"
I20260812 18:59:17.746090 6056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:17.767791 6056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:17.770924 6056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:17.771087 6056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:17.771760 6056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:17.772708 6056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:17.772761 6056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.772819 6056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:17.772841 6056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.782835 6056 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:39297
I20260812 18:59:17.782907 6201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:39297 every 8 connection(s)
I20260812 18:59:17.783993 6056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:17.789687 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 6056
I20260812 18:59:17.789870 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 5330
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.253@KRBTEST.COM
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:17.800706 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.785919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:54919 (local address 127.29.112.253:35525)
0812 18:59:17.786100 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.786106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.787831 (+ 1725us) server_negotiation.cc:407] Connection header received
0812 18:59:17.788166 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.788170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.788224 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.788291 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.789593 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.790420 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.792575 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.792928 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.793008 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.796237 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.796243 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.796247 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.796277 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.798211 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.798902 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.798907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.798909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.798993 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.799425 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.799428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.799430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.799688 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.799764 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.800392 (+ 628us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.800523 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 18:59:17.801246 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.787458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:48375 (local address 127.29.112.254:40813)
0812 18:59:17.787646 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.787652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.787668 (+ 16us) server_negotiation.cc:407] Connection header received
0812 18:59:17.788166 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.788169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.788224 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.788290 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.789593 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.790415 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.794089 (+ 3674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.794420 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.794499 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.797418 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.797424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.797427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.797459 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.799290 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.799832 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.799835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.799836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.799931 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.800223 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.800226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.800228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.800433 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.800500 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.801004 (+ 504us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.801111 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260812 18:59:17.802210 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.786303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.194:48375)
0812 18:59:17.786864 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.786938 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.787946 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.788477 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.788482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.788928 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.789420 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.789440 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.793050 (+ 3610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.793052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.793939 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.793954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.794655 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.794658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.794802 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.795398 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.795413 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.797247 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.799458 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.799460 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.799462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.799726 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.800036 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.800039 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.800041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.800121 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.800568 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.800570 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.800645 (+ 75us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.801506 (+ 861us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.801625 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":79,"spinlock_wait_cycles":4736,"thread_start_us":711,"threads_started":1}
I20260812 18:59:17.802210 6212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.787363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:35525 (local address 127.29.112.194:54919)
0812 18:59:17.787717 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.787730 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.787946 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.788477 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.788482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.788928 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.789420 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.789440 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.790613 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.790619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.792442 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.792454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.793196 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.793200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.793384 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.794053 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.794075 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.796037 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.798400 (+ 2363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.798416 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.798431 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.798786 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.799111 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.799114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.799116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.799285 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.799852 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.799857 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.800137 (+ 280us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.800951 (+ 814us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.801255 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":97,"spinlock_wait_cycles":35072,"thread_start_us":97,"threads_started":1}
I20260812 18:59:17.803565 6206 heartbeater.cc:347] Connected to a master server at 127.29.112.253:35525
I20260812 18:59:17.803565 6203 heartbeater.cc:347] Connected to a master server at 127.29.112.254:40813
I20260812 18:59:17.803817 6206 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:17.803817 6203 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:17.804412 6203 heartbeater.cc:511] Master 127.29.112.254:40813 requested a full tablet report, sending...
I20260812 18:59:17.804419 6206 heartbeater.cc:511] Master 127.29.112.253:35525 requested a full tablet report, sending...
I20260812 18:59:17.805493 4955 ts_manager.cc:194] Re-registered known tserver with Master: b5f9235f90144386bc27513990c1cf8f (127.29.112.194:39297)
I20260812 18:59:17.806183 5639 ts_manager.cc:194] Registered new tserver with Master: b5f9235f90144386bc27513990c1cf8f (127.29.112.194:39297)
I20260812 18:59:17.806244 4955 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:48375
I20260812 18:59:17.815657 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:39415
--local_ip_for_outbound_sockets=127.29.112.195
--tserver_master_addrs=127.29.112.254:40813
--webserver_port=34395
--webserver_interface=127.29.112.195
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab
--principal=oryx/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.112.254:40813,127.29.112.253:35525
--tserver_master_addrs=127.29.112.254:40813,127.29.112.253:35525,127.29.112.252:45951 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.252@KRBTEST.COM
I20260812 18:59:17.821628 6211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.786303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:45951 (local address 127.29.112.194:49271)
0812 18:59:17.787256 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.787269 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.787944 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.788475 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.788481 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.788906 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:17.789436 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.789445 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.814291 (+ 24846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.814296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.814838 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.814846 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.815378 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.815381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.815462 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.816034 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:17.816054 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:17.817719 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:17.819930 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.819932 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.819934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.820124 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.820522 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:17.820524 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:17.820525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:17.820611 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:17.821083 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:17.821116 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:17.821200 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.821289 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.821428 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"mutex_wait_us":66,"spinlock_wait_cycles":19840,"thread_start_us":133,"threads_started":1}
I20260812 18:59:17.821935 6052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.785987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:49271 (local address 127.29.112.252:45951)
0812 18:59:17.786148 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.786153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.787378 (+ 1225us) server_negotiation.cc:407] Connection header received
0812 18:59:17.788143 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.788146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.788200 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.788300 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:17.789603 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.790442 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.814978 (+ 24536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.815217 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.815345 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.817955 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:17.817961 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:17.817966 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:17.818001 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:17.819808 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.820227 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.820233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.820238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.820333 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:17.820702 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:17.820708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:17.820711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:17.820959 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:17.821081 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.821372 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.821536 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 18:59:17.822829 6204 heartbeater.cc:347] Connected to a master server at 127.29.112.252:45951
I20260812 18:59:17.822933 6204 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:17.823145 6204 heartbeater.cc:511] Master 127.29.112.252:45951 requested a full tablet report, sending...
I20260812 18:59:17.823801 5852 ts_manager.cc:194] Registered new tserver with Master: b5f9235f90144386bc27513990c1cf8f (127.29.112.194:39297)
W20260812 18:59:17.924185 6213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:17.924444 6213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:17.924511 6213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:17.928380 6213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:17.928453 6213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:17.928545 6213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:17.933406 6213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab
--principal=oryx/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:39415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=34395
--tserver_master_addrs=127.29.112.254:40813,127.29.112.253:35525,127.29.112.252:45951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:17.934362 6213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:17.935376 6213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 18:59:17.944597 6213 server_base.cc:1034] running on GCE node
W20260812 18:59:17.944540 6222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.944530 6220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:17.944561 6219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:17.945215 6213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:17.945863 6213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:17.947072 6213 hybrid_clock.cc:648] HybridClock initialized: now 1786561157947023 us; error 58 us; skew 500 ppm
Aug 12 18:59:17 dist-test-slave-6nmv krb5kdc[4898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:17.950114 6213 init.cc:377] Logged in from keytab as oryx/127.29.112.195@KRBTEST.COM (short username oryx)
I20260812 18:59:17.950708 6052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.947360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42356 (local address 127.29.112.252:45951)
0812 18:59:17.947507 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:17.947512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:17.947618 (+ 106us) server_negotiation.cc:407] Connection header received
0812 18:59:17.947719 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:17.947725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:17.947796 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:17.947901 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:17.948127 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.948631 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.949638 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.950143 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.950275 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.950396 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:17.950515 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 18:59:17.950599 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 18:59:17.950719 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:17.947288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:45951 (local address 127.0.0.1:42356)
0812 18:59:17.947530 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:17.947542 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:17.947633 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:17.947909 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:17.947911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:17.947925 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:17.948018 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.948022 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.948736 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.948742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.949526 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:17.949536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:17.950294 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:17.950297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:17.950380 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:17.950402 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 18:59:17.950483 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 18:59:17.950580 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
I20260812 18:59:17.951028 6213 webserver.cc:460] Webserver started at http://127.29.112.195:34395/ using document root <none> and password file <none>
I20260812 18:59:17.951534 5864 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 18:59:17.951700 6213 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:17.951800 6213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:17.951997 5894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5d24a92401844d588331250ab11bd13. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:17.952134 5894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: This master's current role is: LEARNER
I20260812 18:59:17.955489 6213 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 18:59:17.957384 6231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:17.958483 6213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:17.958593 6213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "99f725af76044b9db2a594397d1158ee"
format_stamp: "Formatted at 2026-08-12 18:59:16 on dist-test-slave-6nmv"
server_key: "7c651961f2acbf763af81e5417e263e0"
server_key_iv: "ce911ce561db2daa2b25fa8945d6b101"
server_key_version: "encryptionkey@0"
I20260812 18:59:17.959185 6213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:17.999640 6213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:18.002948 6213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:18.003134 6213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:18.003628 6213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:18.004377 6213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:18.004420 6213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:18.004465 6213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:18.004484 6213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:18.013016 6213 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:39415
I20260812 18:59:18.013096 6363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:39415 every 8 connection(s)
I20260812 18:59:18.014097 6213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:59:18.023447 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 6213
Aug 12 18:59:18 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.252@KRBTEST.COM
Aug 12 18:59:18 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.253@KRBTEST.COM
Aug 12 18:59:18 dist-test-slave-6nmv krb5kdc[4898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561157, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:18.030652 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.016333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:45951 (local address 127.29.112.195:38055)
0812 18:59:18.017381 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:18.017394 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:18.018171 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:18.018651 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:18.018656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:18.019156 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:18.019562 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.019571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.020597 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.020600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.021682 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.021691 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.022179 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.022183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.022418 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.023130 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:18.023144 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:18.025382 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:18.027730 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:18.027735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:18.027747 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:18.028069 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:18.028353 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:18.028356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:18.028358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:18.028517 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:18.028941 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:18.028947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:18.029229 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 18:59:18.029422 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 18:59:18.029700 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"mutex_wait_us":287,"spinlock_wait_cycles":25216,"thread_start_us":621,"threads_started":1}
I20260812 18:59:18.031316 6052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.016000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:38055 (local address 127.29.112.252:45951)
0812 18:59:18.016194 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.016197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.017462 (+ 1265us) server_negotiation.cc:407] Connection header received
0812 18:59:18.018380 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.018383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.018436 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.018489 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:18.019702 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.020475 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.021792 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.022045 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.022149 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.025540 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:18.025543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:18.025546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:18.025578 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:18.027603 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.028175 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.028178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.028180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.028261 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.028615 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.028619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.028620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.028812 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:18.028893 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.031017 (+ 2124us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.031145 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260812 18:59:18.031774 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.015963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:51043 (local address 127.29.112.253:35525)
0812 18:59:18.016086 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.016090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.016986 (+ 896us) server_negotiation.cc:407] Connection header received
0812 18:59:18.018357 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.018360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.018417 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.018493 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:18.019689 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.020209 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.021812 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.022163 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.022237 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.026430 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:18.026435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:18.026437 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:18.026466 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:18.028504 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.028997 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.029003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.029007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.029084 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.029421 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.029426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.029430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.030979 (+ 1549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:18.031057 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.031439 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.031555 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 18:59:18.032575 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.016333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:35525 (local address 127.29.112.195:51043)
0812 18:59:18.016840 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:18.016890 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:18.018159 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:18.018648 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:18.018655 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:18.019111 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:18.019547 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.019559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.020348 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.020350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.021683 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.021692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.022381 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.022384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.022508 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.023311 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:18.023322 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:18.026266 (+ 2944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:18.028650 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:18.028652 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:18.028654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:18.028892 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:18.029222 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:18.029224 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:18.029225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:18.029303 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:18.031144 (+ 1841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:18.031146 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:18.031252 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 18:59:18.031954 (+ 702us) client_negotiation.cc:240] Negotiation successful
0812 18:59:18.032089 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":64,"spinlock_wait_cycles":24576,"thread_start_us":133,"threads_started":1}
I20260812 18:59:18.032624 6366 heartbeater.cc:347] Connected to a master server at 127.29.112.252:45951
I20260812 18:59:18.032847 6366 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:18.033144 4998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.015958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:50355 (local address 127.29.112.254:40813)
0812 18:59:18.016084 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.016088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.017437 (+ 1349us) server_negotiation.cc:407] Connection header received
0812 18:59:18.018380 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.018383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.018436 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.018489 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:18.019702 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.020482 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.022388 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.022778 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.022852 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.029255 (+ 6403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:18.029260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:18.029264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:18.029287 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:18.031026 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.031745 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.031748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.031749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.031838 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.032170 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.032174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.032175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.032388 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:18.032483 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.032860 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.032975 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 18:59:18.033159 6368 heartbeater.cc:347] Connected to a master server at 127.29.112.253:35525
I20260812 18:59:18.033319 6368 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:18.033335 6366 heartbeater.cc:511] Master 127.29.112.252:45951 requested a full tablet report, sending...
I20260812 18:59:18.033461 6368 heartbeater.cc:511] Master 127.29.112.253:35525 requested a full tablet report, sending...
I20260812 18:59:18.033596 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.016333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.195:50355)
0812 18:59:18.017302 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:18.017317 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:18.018171 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:18.018651 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:18.018656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:18.019156 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:18.019567 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.019572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.021489 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.021492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.022276 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.022285 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.022897 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.022904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.023013 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.027075 (+ 4062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:18.027106 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:18.029077 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:18.031149 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:18.031151 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:18.031152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:18.031607 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:18.031954 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:18.031957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:18.031959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:18.032044 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:18.032548 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:18.032553 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:18.032652 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 18:59:18.033318 (+ 666us) client_negotiation.cc:240] Negotiation successful
0812 18:59:18.033436 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"mutex_wait_us":151,"spinlock_wait_cycles":94208,"thread_start_us":142,"threads_started":1}
I20260812 18:59:18.034214 6365 heartbeater.cc:347] Connected to a master server at 127.29.112.254:40813
I20260812 18:59:18.034302 6365 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:18.034462 6365 heartbeater.cc:511] Master 127.29.112.254:40813 requested a full tablet report, sending...
I20260812 18:59:18.034862 5852 ts_manager.cc:194] Registered new tserver with Master: 99f725af76044b9db2a594397d1158ee (127.29.112.195:39415)
I20260812 18:59:18.034931 5639 ts_manager.cc:194] Registered new tserver with Master: 99f725af76044b9db2a594397d1158ee (127.29.112.195:39415)
I20260812 18:59:18.035313 4955 ts_manager.cc:194] Re-registered known tserver with Master: 99f725af76044b9db2a594397d1158ee (127.29.112.195:39415)
I20260812 18:59:18.036001 4955 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:50355
I20260812 18:59:18.040639 6380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.026489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43426 (local address 127.29.112.254:40813)
0812 18:59:18.031510 (+ 5021us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.031514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.031764 (+ 250us) server_negotiation.cc:407] Connection header received
0812 18:59:18.031944 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.031949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.031998 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.032069 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:18.033032 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.033667 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.034808 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.035115 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.035180 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.036978 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:18.036982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:18.036985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:18.037009 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:18.038836 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.039294 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.039299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.039302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.039393 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.039724 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.039727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.039729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.039925 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:18.040062 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.040295 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.040413 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":86,"threads_started":1}
I20260812 18:59:18.040673 6378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.025799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.29.112.253:35525)
0812 18:59:18.026065 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.026068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.026580 (+ 512us) server_negotiation.cc:407] Connection header received
0812 18:59:18.026628 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.026631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.026679 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.026764 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:18.027857 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.028807 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.030090 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.030392 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.030477 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.036993 (+ 6516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:18.036997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:18.036999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:18.037030 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:18.038972 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.039474 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.039477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.039479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.039567 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.039847 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.039851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.039855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.040048 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:18.040166 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.040389 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.040511 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
I20260812 18:59:18.045239 6052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.028578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.29.112.252:45951)
0812 18:59:18.031472 (+ 2894us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.031476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.038002 (+ 6526us) server_negotiation.cc:407] Connection header received
0812 18:59:18.038132 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.038136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.038182 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.038286 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:18.039131 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.039807 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.040965 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.041246 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.041315 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.042607 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:18.042611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:18.042616 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:18.042643 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:18.043863 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.044296 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.044298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.044299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.044361 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.044604 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.044606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.044607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.044736 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:18.044820 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.045036 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.045109 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2840,"thread_start_us":79,"threads_started":1}
I20260812 18:59:18.043783 4955 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:59:18.047528 4955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:59:18.050271 6214 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: NON_VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: true } } }
I20260812 18:59:18.050393 6214 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:18.053258 6387 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13: attempting to promote NON_VOTER 93e1da7fa1ee43d0b3c50ec5f003d964 to VOTER
I20260812 18:59:18.053610 6387 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } }
I20260812 18:59:18.054812 5864 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 1 LEARNER]: Refusing update from remote peer d5d24a92401844d588331250ab11bd13: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 18:59:18.054878 5648 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Refusing update from remote peer d5d24a92401844d588331250ab11bd13: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 18:59:18.055197 6387 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 18:59:18.055315 6385 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 18:59:18.057086 6214 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 93e1da7fa1ee43d0b3c50ec5f003d964 (127.29.112.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:18.057545 5648 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 93e1da7fa1ee43d0b3c50ec5f003d964 (127.29.112.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:18.057634 5864 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 93e1da7fa1ee43d0b3c50ec5f003d964 (127.29.112.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:18.058764 6386 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:18.058880 6386 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:18.059028 6386 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:18.059118 6386 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:18.059687 5894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:18.059801 5894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 18:59:18.060164 6395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:18.060326 6395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 18:59:18.058785 6214 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5d24a92401844d588331250ab11bd13" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:18.061395 6214 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:18.070082 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.064835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:41303 (local address 127.0.0.1:48916)
0812 18:59:18.064962 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:18.064977 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:18.065048 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:18.065874 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:18.065877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:18.065895 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:18.065999 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.066016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.067437 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.067440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.068541 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.068551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.069576 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.069580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.069733 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.069756 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 18:59:18.069863 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 18:59:18.069944 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":39}
I20260812 18:59:18.070127 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.065024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.29.112.193:41303)
0812 18:59:18.065357 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.065375 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.065398 (+ 23us) server_negotiation.cc:407] Connection header received
0812 18:59:18.065482 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.065490 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.065699 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.066191 (+ 492us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:18.066349 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.067235 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.068675 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.069455 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.069574 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.069704 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.069894 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.069986 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":97,"threads_started":1}
I20260812 18:59:18.070979 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.065313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:39297 (local address 127.0.0.1:59844)
0812 18:59:18.066219 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:18.066233 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:18.066310 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:18.066959 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:18.066963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:18.066981 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:18.067069 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.067075 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.068563 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.068567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.069669 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.069678 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.070632 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.070635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.070711 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.070731 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 18:59:18.070789 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 18:59:18.070853 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":4,"thread_start_us":99,"threads_started":1}
I20260812 18:59:18.071355 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.065391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39415 (local address 127.0.0.1:34202)
0812 18:59:18.067160 (+ 1769us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:18.067171 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:18.067281 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:18.067710 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:18.067713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:18.067728 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:18.067833 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.067839 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.068869 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.068872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.069805 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.069814 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.071007 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.071010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.071106 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.071125 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 18:59:18.071178 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 18:59:18.071237 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1718,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260812 18:59:18.071648 6399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.065524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.29.112.195:39415)
0812 18:59:18.066291 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.066304 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.067261 (+ 957us) server_negotiation.cc:407] Connection header received
0812 18:59:18.067439 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.067443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.067571 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.067716 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:18.067945 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.068749 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.069952 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.070881 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.071009 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.071265 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.071307 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.071385 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":88,"threads_started":1}
I20260812 18:59:18.073155 5975 tablet_service.cc:1467] Processing CreateTablet for tablet 18da41e2b6cf41679b693864d28cd3cd (DEFAULT_TABLE table=test-table [id=f422a77c46a240edbf98391c5f7ca656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:18.074026 6400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.065858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59844 (local address 127.29.112.194:39297)
0812 18:59:18.066514 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.066523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.066541 (+ 18us) server_negotiation.cc:407] Connection header received
0812 18:59:18.066601 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.066604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.066807 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.066960 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:18.067204 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.068421 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.069813 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.070513 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.070644 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.070757 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.070810 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.070883 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":93,"threads_started":1}
I20260812 18:59:18.074283 5975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18da41e2b6cf41679b693864d28cd3cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:18.074420 6287 tablet_service.cc:1467] Processing CreateTablet for tablet 18da41e2b6cf41679b693864d28cd3cd (DEFAULT_TABLE table=test-table [id=f422a77c46a240edbf98391c5f7ca656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:18.075346 6287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18da41e2b6cf41679b693864d28cd3cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:18.076294 6131 tablet_service.cc:1467] Processing CreateTablet for tablet 18da41e2b6cf41679b693864d28cd3cd (DEFAULT_TABLE table=test-table [id=f422a77c46a240edbf98391c5f7ca656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:18.077049 6131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18da41e2b6cf41679b693864d28cd3cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:18.083050 6401 tablet_bootstrap.cc:492] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: Bootstrap starting.
I20260812 18:59:18.083488 6403 tablet_bootstrap.cc:492] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47: Bootstrap starting.
I20260812 18:59:18.084193 6405 tablet_bootstrap.cc:492] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f: Bootstrap starting.
I20260812 18:59:18.086385 6401 tablet_bootstrap.cc:654] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:18.086570 6405 tablet_bootstrap.cc:654] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:18.087374 6401 log.cc:824] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:18.087374 6405 log.cc:824] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:18.088047 6403 tablet_bootstrap.cc:654] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:18.088977 6403 log.cc:824] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:18.089339 6405 tablet_bootstrap.cc:492] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f: No bootstrap required, opened a new log
I20260812 18:59:18.089537 6405 ts_tablet_manager.cc:1397] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 18:59:18.091109 6401 tablet_bootstrap.cc:492] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: No bootstrap required, opened a new log
I20260812 18:59:18.091189 6403 tablet_bootstrap.cc:492] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47: No bootstrap required, opened a new log
I20260812 18:59:18.091265 6401 ts_tablet_manager.cc:1397] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 18:59:18.091334 6403 ts_tablet_manager.cc:1397] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 18:59:18.094460 6405 raft_consensus.cc:348] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } }
I20260812 18:59:18.094712 6405 raft_consensus.cc:374] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:18.094785 6405 raft_consensus.cc:729] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f9235f90144386bc27513990c1cf8f, State: Initialized, Role: FOLLOWER
I20260812 18:59:18.095293 6405 consensus_queue.cc:260] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } }
I20260812 18:59:18.096324 6405 ts_tablet_manager.cc:1428] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 18:59:18.096225 6403 raft_consensus.cc:348] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } }
I20260812 18:59:18.096442 6403 raft_consensus.cc:374] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:18.096505 6403 raft_consensus.cc:729] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17baa51a90904d65b6a9a07958887c47, State: Initialized, Role: FOLLOWER
I20260812 18:59:18.096635 6203 heartbeater.cc:503] Master 127.29.112.254:40813 was elected leader, sending a full tablet report...
I20260812 18:59:18.096873 6403 consensus_queue.cc:260] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } }
I20260812 18:59:18.096872 6401 raft_consensus.cc:348] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } }
I20260812 18:59:18.097218 6401 raft_consensus.cc:374] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:18.097325 6401 raft_consensus.cc:729] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f725af76044b9db2a594397d1158ee, State: Initialized, Role: FOLLOWER
I20260812 18:59:18.097724 6403 ts_tablet_manager.cc:1428] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 18:59:18.097895 6048 heartbeater.cc:503] Master 127.29.112.254:40813 was elected leader, sending a full tablet report...
I20260812 18:59:18.097765 6401 consensus_queue.cc:260] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } }
I20260812 18:59:18.099031 6401 ts_tablet_manager.cc:1428] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260812 18:59:18.099390 6365 heartbeater.cc:503] Master 127.29.112.254:40813 was elected leader, sending a full tablet report...
I20260812 18:59:18.240316 6410 raft_consensus.cc:482] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:59:18.240574 6410 raft_consensus.cc:504] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } }
I20260812 18:59:18.241719 6410 leader_election.cc:288] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17baa51a90904d65b6a9a07958887c47 (127.29.112.193:41303), 99f725af76044b9db2a594397d1158ee (127.29.112.195:39415)
I20260812 18:59:18.245337 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.242128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:42035 (local address 127.29.112.193:41303)
0812 18:59:18.242246 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.242251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.242266 (+ 15us) server_negotiation.cc:407] Connection header received
0812 18:59:18.242331 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.242334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.242383 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.242473 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:18.242733 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.243238 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.244345 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.244733 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.244873 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.244940 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.245143 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.245211 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 18:59:18.245347 6211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.241982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:41303 (local address 127.29.112.194:42035)
0812 18:59:18.242128 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:18.242144 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:18.242245 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:18.242487 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:18.242490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:18.242506 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:18.242620 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.242625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.243353 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.243357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.244218 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.244227 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.244867 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.244870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.245004 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.245042 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 18:59:18.245128 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 18:59:18.245201 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":19}
I20260812 18:59:18.245494 6399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.242119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:53045 (local address 127.29.112.195:39415)
0812 18:59:18.242257 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.242262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.242409 (+ 147us) server_negotiation.cc:407] Connection header received
0812 18:59:18.242572 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.242574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.242612 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.242692 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:18.242935 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.243432 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.244423 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.244818 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.244961 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.245020 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.245296 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.245361 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 18:59:18.245494 6212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.241982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39415 (local address 127.29.112.194:53045)
0812 18:59:18.242332 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:18.242343 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:18.242446 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:18.242701 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:18.242704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:18.242718 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:18.242816 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.242824 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.243557 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.243562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.244300 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:18.244308 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.244962 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.244965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:18.245160 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.245180 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 18:59:18.245279 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 18:59:18.245342 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":226}
I20260812 18:59:18.246181 6310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18da41e2b6cf41679b693864d28cd3cd" candidate_uuid: "b5f9235f90144386bc27513990c1cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99f725af76044b9db2a594397d1158ee" is_pre_election: true
I20260812 18:59:18.246182 5997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18da41e2b6cf41679b693864d28cd3cd" candidate_uuid: "b5f9235f90144386bc27513990c1cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17baa51a90904d65b6a9a07958887c47" is_pre_election: true
I20260812 18:59:18.246451 5997 raft_consensus.cc:2454] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f9235f90144386bc27513990c1cf8f in term 0.
I20260812 18:59:18.246466 6310 raft_consensus.cc:2454] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f9235f90144386bc27513990c1cf8f in term 0.
I20260812 18:59:18.246893 6074 leader_election.cc:302] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17baa51a90904d65b6a9a07958887c47, b5f9235f90144386bc27513990c1cf8f; no voters:
I20260812 18:59:18.247175 6410 raft_consensus.cc:2781] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:59:18.247291 6410 raft_consensus.cc:482] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:59:18.247359 6410 raft_consensus.cc:3037] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:18.248402 6410 raft_consensus.cc:504] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } }
I20260812 18:59:18.248786 6410 leader_election.cc:288] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [CANDIDATE]: Term 1 election: Requested vote from peers 17baa51a90904d65b6a9a07958887c47 (127.29.112.193:41303), 99f725af76044b9db2a594397d1158ee (127.29.112.195:39415)
I20260812 18:59:18.249076 5997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18da41e2b6cf41679b693864d28cd3cd" candidate_uuid: "b5f9235f90144386bc27513990c1cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17baa51a90904d65b6a9a07958887c47"
I20260812 18:59:18.249104 6310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18da41e2b6cf41679b693864d28cd3cd" candidate_uuid: "b5f9235f90144386bc27513990c1cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99f725af76044b9db2a594397d1158ee"
I20260812 18:59:18.249187 5997 raft_consensus.cc:3037] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:18.249214 6310 raft_consensus.cc:3037] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:18.250361 5997 raft_consensus.cc:2454] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f9235f90144386bc27513990c1cf8f in term 1.
I20260812 18:59:18.250530 6310 raft_consensus.cc:2454] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f9235f90144386bc27513990c1cf8f in term 1.
I20260812 18:59:18.250610 6074 leader_election.cc:302] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17baa51a90904d65b6a9a07958887c47, b5f9235f90144386bc27513990c1cf8f; no voters:
I20260812 18:59:18.250787 6410 raft_consensus.cc:2781] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:18.250974 6410 raft_consensus.cc:686] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 1 LEADER]: Becoming Leader. State: Replica: b5f9235f90144386bc27513990c1cf8f, State: Running, Role: LEADER
I20260812 18:59:18.251309 6410 consensus_queue.cc:237] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } }
I20260812 18:59:18.254510 4955 catalog_manager.cc:5495] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f reported cstate change: term changed from 0 to 1, leader changed from <none> to b5f9235f90144386bc27513990c1cf8f (127.29.112.194). New cstate: current_term: 1 leader_uuid: "b5f9235f90144386bc27513990c1cf8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } health_report { overall_health: HEALTHY } } }
I20260812 18:59:18.266215 6400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.263463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59856 (local address 127.29.112.194:39297)
0812 18:59:18.263565 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.263567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.263578 (+ 11us) server_negotiation.cc:407] Connection header received
0812 18:59:18.263631 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.263632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.263674 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.263750 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:59:18.263920 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.264353 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.265065 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.265242 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.265306 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.265519 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:59:18.265903 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:59:18.265989 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.266043 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.266102 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 18:59:18.267580 6369 tablet.cc:2369] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:59:18.271790 6310 raft_consensus.cc:1261] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee [term 1 FOLLOWER]: Refusing update from remote peer b5f9235f90144386bc27513990c1cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:18.271826 5997 raft_consensus.cc:1261] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47 [term 1 FOLLOWER]: Refusing update from remote peer b5f9235f90144386bc27513990c1cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:18.272513 6413 consensus_queue.cc:1035] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:18.272682 6410 consensus_queue.cc:1035] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:18.279953 6416 mvcc.cc:205] Tried to move back new op lower bound from 7317754504274939904 to 7317754504199958528. Current Snapshot: MvccSnapshot[applied={T|T < 7317754504274939904}]
I20260812 18:59:18.284888 6417 mvcc.cc:205] Tried to move back new op lower bound from 7317754504274939904 to 7317754504199958528. Current Snapshot: MvccSnapshot[applied={T|T < 7317754504274939904 or (T in {7317754504274939904})}]
I20260812 18:59:18.285279 6418 mvcc.cc:205] Tried to move back new op lower bound from 7317754504274939904 to 7317754504199958528. Current Snapshot: MvccSnapshot[applied={T|T < 7317754504274939904 or (T in {7317754504274939904})}]
I20260812 18:59:18.290527 4953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:18.290671 4953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:18.294605 4953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5d24a92401844d588331250ab11bd13: Sending DeleteTablet for 3 replicas of tablet 18da41e2b6cf41679b693864d28cd3cd
I20260812 18:59:18.295189 5975 tablet_service.cc:1514] Processing DeleteTablet for tablet 18da41e2b6cf41679b693864d28cd3cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:18 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:48916
I20260812 18:59:18.295574 6131 tablet_service.cc:1514] Processing DeleteTablet for tablet 18da41e2b6cf41679b693864d28cd3cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:18 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:59844
I20260812 18:59:18.295670 6427 tablet_replica.cc:330] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47: stopping tablet replica
I20260812 18:59:18.295785 6287 tablet_service.cc:1514] Processing DeleteTablet for tablet 18da41e2b6cf41679b693864d28cd3cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:18 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:34202
I20260812 18:59:18.295933 6427 raft_consensus.cc:2229] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:18.296231 6427 raft_consensus.cc:2258] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:18.296494 6430 tablet_replica.cc:330] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: stopping tablet replica
I20260812 18:59:18.297148 6430 raft_consensus.cc:2229] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:18.297442 6430 raft_consensus.cc:2258] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:18.297715 6427 ts_tablet_manager.cc:1905] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:18.298842 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 4912
I20260812 18:59:18.302969 6427 ts_tablet_manager.cc:1918] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:18.303071 6427 log.cc:1196] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-0/wal/wals/18da41e2b6cf41679b693864d28cd3cd
I20260812 18:59:18.303366 6430 ts_tablet_manager.cc:1905] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:18.303481 6427 ts_tablet_manager.cc:1939] T 18da41e2b6cf41679b693864d28cd3cd P 17baa51a90904d65b6a9a07958887c47: Deleting consensus metadata
I20260812 18:59:18.306452 6429 tablet_replica.cc:330] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f: stopping tablet replica
I20260812 18:59:18.306734 6429 raft_consensus.cc:2229] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:59:18.307047 6429 raft_consensus.cc:2258] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:18.308176 6430 ts_tablet_manager.cc:1918] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:18.308297 6429 ts_tablet_manager.cc:1905] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:18.311005 6429 ts_tablet_manager.cc:1918] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:18.311115 6429 log.cc:1196] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/18da41e2b6cf41679b693864d28cd3cd
I20260812 18:59:18.311232 6430 log.cc:1196] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/18da41e2b6cf41679b693864d28cd3cd
I20260812 18:59:18.311411 6429 ts_tablet_manager.cc:1939] T 18da41e2b6cf41679b693864d28cd3cd P b5f9235f90144386bc27513990c1cf8f: Deleting consensus metadata
I20260812 18:59:18.311693 6430 ts_tablet_manager.cc:1939] T 18da41e2b6cf41679b693864d28cd3cd P 99f725af76044b9db2a594397d1158ee: Deleting consensus metadata
W20260812 18:59:18.312409 6076 connection.cc:535] server connection from 127.0.0.1:59844 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 18:59:18.312455 5926 connection.cc:535] server connection from 127.0.0.1:48916 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 18:59:18.313477 6235 connection.cc:441] server connection from 127.0.0.1:34202 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34202 (request call id 1) could send its response
I20260812 18:59:18.318572 6378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.298167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47668 (local address 127.29.112.253:35525)
0812 18:59:18.298260 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.298264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.306167 (+ 7903us) server_negotiation.cc:407] Connection header received
0812 18:59:18.306308 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.306314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.306374 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.306463 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:18.309199 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.310057 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.312941 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.313275 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.313370 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.315094 (+ 1724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:18.315097 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:18.315100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:18.315125 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:18.316764 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.317242 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.317245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.317247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.317340 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.317616 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.317619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.317621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.317867 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:18.317970 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.318300 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.318415 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 18:59:18.325052 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.316490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42380 (local address 127.29.112.252:45951)
0812 18:59:18.316632 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.316637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.316819 (+ 182us) server_negotiation.cc:407] Connection header received
0812 18:59:18.316939 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.316943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.316989 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.317088 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:18.317951 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.318546 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.319577 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.319863 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.319919 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.321116 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:18.321119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:18.321121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:18.321143 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:18.323029 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.323598 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.323602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.323606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.323694 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.323978 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.323980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.323982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.324173 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:18.324285 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.324748 (+ 463us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.324863 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 18:59:18.327493 6052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.297878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42368 (local address 127.29.112.252:45951)
0812 18:59:18.297974 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.297978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.319231 (+ 21253us) server_negotiation.cc:407] Connection header received
0812 18:59:18.319373 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.319375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.319411 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.319453 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:18.320197 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.321093 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.322150 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.322426 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.322488 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.323996 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:18.323999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:18.324001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:18.324024 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:18.325542 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.325985 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.325988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.325990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.326062 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.326382 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.326386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.326388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.326576 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:18.326958 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.327204 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.327318 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 18:59:18.330799 6378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:18.322233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47680 (local address 127.29.112.253:35525)
0812 18:59:18.322332 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:18.322337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:18.323158 (+ 821us) server_negotiation.cc:407] Connection header received
0812 18:59:18.323456 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:18.323459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:18.323495 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:18.323547 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:18.324246 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.324889 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.325896 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:18.326198 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:18.326266 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:18.327731 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:18.327734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:18.327737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:18.327759 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:18.329306 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.329708 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.329711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.329713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.329788 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:18.330096 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:18.330099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:18.330101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:18.330276 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:18.330385 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:18.330575 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 18:59:18.330669 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 18:59:19.287096 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.286425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.194:57905)
0812 18:59:19.286737 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.286835 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":68,"threads_started":1}
I20260812 18:59:19.287127 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.286240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.195:36395)
0812 18:59:19.286931 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.287025 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":80,"threads_started":1}
W20260812 18:59:19.287428 6365 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:40813 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
I20260812 18:59:19.287487 6445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.286985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.193:51189)
0812 18:59:19.287275 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.287367 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":63,"threads_started":1}
W20260812 18:59:19.287581 6203 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:40813 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
W20260812 18:59:19.287735 6048 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:40813 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
I20260812 18:59:19.795603 6448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d5d24a92401844d588331250ab11bd13)
I20260812 18:59:19.795819 6448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } }
I20260812 18:59:19.795938 6449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d5d24a92401844d588331250ab11bd13)
I20260812 18:59:19.796142 6449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } }
I20260812 18:59:19.796864 6448 leader_election.cc:288] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813), 1308435398344058b8a7435586e31d70 (127.29.112.253:35525)
I20260812 18:59:19.797616 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.797040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43918)
0812 18:59:19.797426 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.797520 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":62,"threads_started":1}
W20260812 18:59:19.797797 5835 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
I20260812 18:59:19.797842 6449 leader_election.cc:288] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813), 93e1da7fa1ee43d0b3c50ec5f003d964 (127.29.112.252:45951)
I20260812 18:59:19.798663 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.797333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43934)
0812 18:59:19.797999 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.798068 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":66,"threads_started":1}
W20260812 18:59:19.798875 5625 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
I20260812 18:59:19.798985 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.798625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43950)
0812 18:59:19.798880 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.798926 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 18:59:19.799155 5835 leader_election.cc:334] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813): Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
I20260812 18:59:19.799929 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.799708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43964)
0812 18:59:19.799819 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.799866 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 18:59:19.800094 5625 leader_election.cc:334] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813): Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
I20260812 18:59:19.802182 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.797192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:45951 (local address 127.0.0.1:42394)
0812 18:59:19.797654 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.797668 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:19.797760 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:19.798179 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:19.798182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:19.798198 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:19.798299 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:19.798305 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:19.799705 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:19.799708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:19.800932 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:19.800942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:19.801718 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:19.801721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:19.801788 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:19.801860 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 18:59:19.801968 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 18:59:19.802051 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":67,"threads_started":1}
I20260812 18:59:19.802191 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.797699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.29.112.252:45951)
0812 18:59:19.797968 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:19.797972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:19.797985 (+ 13us) server_negotiation.cc:407] Connection header received
0812 18:59:19.798031 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:19.798034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:19.798080 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:19.798171 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:19.798519 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:19.799596 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:19.801049 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:19.801593 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:19.801706 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:19.801757 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:19.801987 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 18:59:19.802064 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":188,"thread_start_us":72,"threads_started":1}
I20260812 18:59:19.802414 6453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.797192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47692 (local address 127.29.112.253:35525)
0812 18:59:19.798187 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:19.798193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:19.798209 (+ 16us) server_negotiation.cc:407] Connection header received
0812 18:59:19.798254 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:19.798257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:19.798308 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:19.798465 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:19.799167 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:19.799880 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:19.801284 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:19.801875 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:19.802001 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:19.802054 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:19.802213 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 18:59:19.802304 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":931,"thread_start_us":69,"threads_started":1}
I20260812 18:59:19.802436 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.797079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:35525 (local address 127.0.0.1:47692)
0812 18:59:19.797426 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.797443 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:19.797599 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:19.798493 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:19.798495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:19.798514 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:19.798720 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:19.798729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:19.800041 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:19.800045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:19.801178 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:19.801187 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:19.801996 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:19.801999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:19.802086 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:19.802109 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 18:59:19.802186 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 18:59:19.802265 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":91,"thread_start_us":71,"threads_started":1}
I20260812 18:59:19.802811 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1308435398344058b8a7435586e31d70" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" is_pre_election: true
I20260812 18:59:19.802866 5648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1308435398344058b8a7435586e31d70" is_pre_election: true
I20260812 18:59:19.803051 5864 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1308435398344058b8a7435586e31d70 in term 1.
I20260812 18:59:19.803089 5648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93e1da7fa1ee43d0b3c50ec5f003d964 in term 1.
I20260812 18:59:19.803438 5626 leader_election.cc:302] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1308435398344058b8a7435586e31d70, 93e1da7fa1ee43d0b3c50ec5f003d964; no voters: d5d24a92401844d588331250ab11bd13
I20260812 18:59:19.803465 5834 leader_election.cc:302] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1308435398344058b8a7435586e31d70, 93e1da7fa1ee43d0b3c50ec5f003d964; no voters: d5d24a92401844d588331250ab11bd13
I20260812 18:59:19.803653 6448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 18:59:19.803745 6448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d5d24a92401844d588331250ab11bd13)
I20260812 18:59:19.803754 6449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 18:59:19.803800 6448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 1 FOLLOWER]: Advancing to term 2
I20260812 18:59:19.803807 6449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d5d24a92401844d588331250ab11bd13)
I20260812 18:59:19.803851 6449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 1 FOLLOWER]: Advancing to term 2
I20260812 18:59:19.805068 6448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } }
I20260812 18:59:19.805068 6449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } }
I20260812 18:59:19.805418 6448 leader_election.cc:288] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [CANDIDATE]: Term 2 election: Requested vote from peers d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813), 1308435398344058b8a7435586e31d70 (127.29.112.253:35525)
I20260812 18:59:19.805430 6449 leader_election.cc:288] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 2 election: Requested vote from peers d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813), 93e1da7fa1ee43d0b3c50ec5f003d964 (127.29.112.252:45951)
I20260812 18:59:19.805858 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.805536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43966)
0812 18:59:19.805673 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.805722 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 18:59:19.805737 5648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1308435398344058b8a7435586e31d70"
I20260812 18:59:19.805927 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.805649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43972)
0812 18:59:19.805791 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.805857 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 18:59:19.805960 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1308435398344058b8a7435586e31d70" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964"
I20260812 18:59:19.806022 5648 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 93e1da7fa1ee43d0b3c50ec5f003d964 in current term 2: Already voted for candidate 1308435398344058b8a7435586e31d70 in this term.
I20260812 18:59:19.806118 5864 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1308435398344058b8a7435586e31d70 in current term 2: Already voted for candidate 93e1da7fa1ee43d0b3c50ec5f003d964 in this term.
I20260812 18:59:19.806632 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.806377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43978)
0812 18:59:19.806525 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.806571 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260812 18:59:19.806759 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:19.806540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:43992)
0812 18:59:19.806638 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:19.806681 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 18:59:19.806752 5835 leader_election.cc:334] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813): Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
I20260812 18:59:19.806819 5835 leader_election.cc:302] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93e1da7fa1ee43d0b3c50ec5f003d964; no voters: 1308435398344058b8a7435586e31d70, d5d24a92401844d588331250ab11bd13
W20260812 18:59:19.806877 5625 leader_election.cc:334] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813): Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
I20260812 18:59:19.806941 5625 leader_election.cc:302] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1308435398344058b8a7435586e31d70; no voters: 93e1da7fa1ee43d0b3c50ec5f003d964, d5d24a92401844d588331250ab11bd13
I20260812 18:59:19.806959 6448 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 18:59:19.807085 6449 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 18:59:20.288447 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:20.288014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.194:48691)
0812 18:59:20.288265 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:20.288337 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":68,"threads_started":1}
I20260812 18:59:20.288736 6458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:20.287967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.195:36359)
0812 18:59:20.288593 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:20.288650 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":96,"threads_started":1}
I20260812 18:59:20.288962 6463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:20.288341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.193:34231)
0812 18:59:20.288825 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:20.288884 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":72,"threads_started":1}
I20260812 18:59:21.289935 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.289346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.194:49493)
0812 18:59:21.289700 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.289800 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":88,"threads_started":1}
I20260812 18:59:21.290086 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.289591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.193:44165)
0812 18:59:21.289890 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.289980 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":61,"threads_started":1}
I20260812 18:59:21.290364 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.289346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.195:46357)
0812 18:59:21.290221 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.290277 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":803,"thread_start_us":83,"threads_started":1}
I20260812 18:59:21.602789 6468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:59:21.602941 6468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } }
I20260812 18:59:21.603367 6468 leader_election.cc:288] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813), 93e1da7fa1ee43d0b3c50ec5f003d964 (127.29.112.252:45951)
I20260812 18:59:21.603892 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.603549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:44256)
0812 18:59:21.603760 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.603814 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260812 18:59:21.603849 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1308435398344058b8a7435586e31d70" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" is_pre_election: true
I20260812 18:59:21.604023 5864 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1308435398344058b8a7435586e31d70 in term 2.
I20260812 18:59:21.604399 5626 leader_election.cc:302] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1308435398344058b8a7435586e31d70, 93e1da7fa1ee43d0b3c50ec5f003d964; no voters:
I20260812 18:59:21.604595 6468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 18:59:21.604717 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.604439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:44272)
0812 18:59:21.604554 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.604601 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 18:59:21.604707 6468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:59:21.604805 6468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 2 FOLLOWER]: Advancing to term 3
W20260812 18:59:21.604833 5625 leader_election.cc:334] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813): Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
I20260812 18:59:21.605867 6468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } }
I20260812 18:59:21.606197 6468 leader_election.cc:288] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 3 election: Requested vote from peers d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813), 93e1da7fa1ee43d0b3c50ec5f003d964 (127.29.112.252:45951)
I20260812 18:59:21.606447 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.606268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:44284)
0812 18:59:21.606337 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.606389 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 18:59:21.606609 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1308435398344058b8a7435586e31d70" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964"
I20260812 18:59:21.606762 5864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 2 FOLLOWER]: Advancing to term 3
I20260812 18:59:21.607091 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.606833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:44298)
0812 18:59:21.606957 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.607026 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 18:59:21.607220 5625 leader_election.cc:334] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813): Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
I20260812 18:59:21.607963 5864 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1308435398344058b8a7435586e31d70 in term 3.
I20260812 18:59:21.608250 5626 leader_election.cc:302] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1308435398344058b8a7435586e31d70, 93e1da7fa1ee43d0b3c50ec5f003d964; no voters: d5d24a92401844d588331250ab11bd13
I20260812 18:59:21.608435 6468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 18:59:21.608639 6468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [term 3 LEADER]: Becoming Leader. State: Replica: 1308435398344058b8a7435586e31d70, State: Running, Role: LEADER
I20260812 18:59:21.608882 6468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } }
I20260812 18:59:21.609810 6472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1308435398344058b8a7435586e31d70. Latest consensus state: current_term: 3 leader_uuid: "1308435398344058b8a7435586e31d70" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:21.609961 6472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:21.610268 6474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:21.613173 6474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:21.613718 6474 catalog_manager.cc:1259] Loaded cluster ID: 85d01156aff54b33b86e23f078b6ba6e
I20260812 18:59:21.613771 6474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:21.614406 6474 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:21.614825 6474 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: Loaded TSK: 0
I20260812 18:59:21.615509 6474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 18:59:21.702885 5639 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:59:21.705889 5639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:59:21.708742 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.708519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:44320)
0812 18:59:21.708628 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.708681 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 18:59:21.708804 5864 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [term 3 FOLLOWER]: Refusing update from remote peer 1308435398344058b8a7435586e31d70: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 18:59:21.709286 6472 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 18:59:21.709388 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.709195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.0.0.1:44334)
0812 18:59:21.709291 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.709333 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 18:59:21.709515 5625 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 -> Peer d5d24a92401844d588331250ab11bd13 (127.29.112.254:40813): Couldn't send request to peer d5d24a92401844d588331250ab11bd13. Status: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 18:59:21.710584 6476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1308435398344058b8a7435586e31d70. Latest consensus state: current_term: 3 leader_uuid: "1308435398344058b8a7435586e31d70" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:21.710702 6476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 18:59:21.711767 6472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1308435398344058b8a7435586e31d70" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:21.711941 6472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:21.712221 6476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1308435398344058b8a7435586e31d70" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d5d24a92401844d588331250ab11bd13" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40813 } } peers { permanent_uuid: "1308435398344058b8a7435586e31d70" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 35525 } attrs { promote: false } } peers { permanent_uuid: "93e1da7fa1ee43d0b3c50ec5f003d964" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 45951 } attrs { promote: false } } }
I20260812 18:59:21.712358 6476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e1da7fa1ee43d0b3c50ec5f003d964 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 18:59:21.721779 6482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.717755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:39297 (local address 127.0.0.1:59868)
0812 18:59:21.718011 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.718023 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:21.718118 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:21.718726 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:21.718729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:21.718746 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:21.718864 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:21.718874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.719799 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.719802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:21.720493 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:21.720501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.721390 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.721394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:21.721480 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:21.721502 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 18:59:21.721581 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 18:59:21.721658 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":130,"threads_started":1}
I20260812 18:59:21.722421 6483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.718146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59868 (local address 127.29.112.194:39297)
0812 18:59:21.718407 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:21.718411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:21.718424 (+ 13us) server_negotiation.cc:407] Connection header received
0812 18:59:21.718522 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:21.718526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:21.718575 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:21.718977 (+ 402us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:21.719087 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.719667 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.720664 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.721271 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.721403 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:21.721460 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:21.722254 (+ 794us) server_negotiation.cc:299] Negotiation successful
0812 18:59:21.722328 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":76,"threads_started":1}
I20260812 18:59:21.722762 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.717477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:41303 (local address 127.0.0.1:48932)
0812 18:59:21.717923 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.717937 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:21.718032 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:21.719025 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:21.719027 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:21.719042 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:21.719137 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:21.719143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.720071 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.720074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:21.721186 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:21.721195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.722379 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.722382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:21.722476 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:21.722533 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 18:59:21.722583 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 18:59:21.722648 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373}
I20260812 18:59:21.722980 6131 tablet_service.cc:1467] Processing CreateTablet for tablet 924fc38e11d747609a75c6adce3053c4 (DEFAULT_TABLE table=test-table [id=ba37910063cb4f56ac22f99e19e07d70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:21.723201 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.718669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39415 (local address 127.0.0.1:34204)
0812 18:59:21.718932 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.718946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:21.719021 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:21.719491 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:21.719494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:21.719510 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:21.719592 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:21.719597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.720626 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.720629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:21.721671 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:21.721679 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.722820 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.722823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:21.722894 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:21.722916 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 18:59:21.722993 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 18:59:21.723079 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260812 18:59:21.723323 6131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 924fc38e11d747609a75c6adce3053c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:21.723248 6485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.718790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34204 (local address 127.29.112.195:39415)
0812 18:59:21.719250 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:21.719254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:21.719267 (+ 13us) server_negotiation.cc:407] Connection header received
0812 18:59:21.719314 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:21.719316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:21.719360 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:21.719691 (+ 331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:21.719810 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.720511 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.721829 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.722450 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.722826 (+ 376us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:21.722879 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:21.723040 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 18:59:21.723119 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":64,"threads_started":1}
I20260812 18:59:21.723513 6481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.717649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.29.112.193:41303)
0812 18:59:21.718775 (+ 1126us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:21.718779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:21.718794 (+ 15us) server_negotiation.cc:407] Connection header received
0812 18:59:21.718845 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:21.718849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:21.718900 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:21.719011 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:21.719251 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.719944 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.721605 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.722255 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.723217 (+ 962us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:21.723265 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:21.723304 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 18:59:21.723384 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":113,"threads_started":1}
I20260812 18:59:21.723973 6287 tablet_service.cc:1467] Processing CreateTablet for tablet 924fc38e11d747609a75c6adce3053c4 (DEFAULT_TABLE table=test-table [id=ba37910063cb4f56ac22f99e19e07d70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:21.724021 5975 tablet_service.cc:1467] Processing CreateTablet for tablet 924fc38e11d747609a75c6adce3053c4 (DEFAULT_TABLE table=test-table [id=ba37910063cb4f56ac22f99e19e07d70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:21.724174 6287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 924fc38e11d747609a75c6adce3053c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:21.724263 5975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 924fc38e11d747609a75c6adce3053c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:21.726572 6486 tablet_bootstrap.cc:492] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee: Bootstrap starting.
I20260812 18:59:21.726917 6488 tablet_bootstrap.cc:492] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47: Bootstrap starting.
I20260812 18:59:21.727367 6487 tablet_bootstrap.cc:492] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f: Bootstrap starting.
I20260812 18:59:21.727614 6486 tablet_bootstrap.cc:654] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:21.727839 6488 tablet_bootstrap.cc:654] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:21.728293 6487 tablet_bootstrap.cc:654] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:21.728876 6486 tablet_bootstrap.cc:492] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee: No bootstrap required, opened a new log
I20260812 18:59:21.728993 6486 ts_tablet_manager.cc:1397] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:21.729432 6487 tablet_bootstrap.cc:492] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f: No bootstrap required, opened a new log
I20260812 18:59:21.729434 6486 raft_consensus.cc:348] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } }
I20260812 18:59:21.729564 6487 ts_tablet_manager.cc:1397] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:21.729601 6486 raft_consensus.cc:374] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:21.729655 6486 raft_consensus.cc:729] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f725af76044b9db2a594397d1158ee, State: Initialized, Role: FOLLOWER
I20260812 18:59:21.729787 6486 consensus_queue.cc:260] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } }
I20260812 18:59:21.730041 6487 raft_consensus.cc:348] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } }
I20260812 18:59:21.730168 6487 raft_consensus.cc:374] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:21.730237 6487 raft_consensus.cc:729] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f9235f90144386bc27513990c1cf8f, State: Initialized, Role: FOLLOWER
I20260812 18:59:21.730290 6486 ts_tablet_manager.cc:1428] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 18:59:21.730371 6487 consensus_queue.cc:260] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } }
I20260812 18:59:21.730613 6488 tablet_bootstrap.cc:492] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47: No bootstrap required, opened a new log
I20260812 18:59:21.730700 6488 ts_tablet_manager.cc:1397] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 18:59:21.730818 6487 ts_tablet_manager.cc:1428] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:21.731119 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.730792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.195:53759)
0812 18:59:21.730974 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.731047 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260812 18:59:21.731164 6488 raft_consensus.cc:348] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } }
I20260812 18:59:21.731267 6488 raft_consensus.cc:374] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:21.731298 6488 raft_consensus.cc:729] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17baa51a90904d65b6a9a07958887c47, State: Initialized, Role: FOLLOWER
I20260812 18:59:21.731410 6488 consensus_queue.cc:260] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } }
I20260812 18:59:21.731814 6488 ts_tablet_manager.cc:1428] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:21.732838 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.732592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.195:39861)
0812 18:59:21.732710 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.732768 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 18:59:21.732980 6368 heartbeater.cc:503] Master 127.29.112.253:35525 was elected leader, sending a full tablet report...
I20260812 18:59:21.733729 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.732433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.194:57561)
0812 18:59:21.733610 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.733663 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1102}
I20260812 18:59:21.733855 6206 heartbeater.cc:503] Master 127.29.112.253:35525 was elected leader, sending a full tablet report...
I20260812 18:59:21.735150 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.732701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.193:37679)
0812 18:59:21.735036 (+ 2335us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.735088 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2244}
I20260812 18:59:21.735750 6050 heartbeater.cc:503] Master 127.29.112.253:35525 was elected leader, sending a full tablet report...
I20260812 18:59:21.735957 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.735734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.194:38403)
0812 18:59:21.735842 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.735887 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 18:59:21.736737 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.736488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.193:60617)
0812 18:59:21.736622 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.736677 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 18:59:21.788949 6208 tablet.cc:2369] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:59:21.801072 6495 raft_consensus.cc:482] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:59:21.801270 6495 raft_consensus.cc:504] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } }
I20260812 18:59:21.801677 6495 leader_election.cc:288] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17baa51a90904d65b6a9a07958887c47 (127.29.112.193:41303), 99f725af76044b9db2a594397d1158ee (127.29.112.195:39415)
I20260812 18:59:21.802135 5997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "924fc38e11d747609a75c6adce3053c4" candidate_uuid: "b5f9235f90144386bc27513990c1cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17baa51a90904d65b6a9a07958887c47" is_pre_election: true
I20260812 18:59:21.802214 6310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "924fc38e11d747609a75c6adce3053c4" candidate_uuid: "b5f9235f90144386bc27513990c1cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99f725af76044b9db2a594397d1158ee" is_pre_election: true
I20260812 18:59:21.802321 5997 raft_consensus.cc:2454] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f9235f90144386bc27513990c1cf8f in term 0.
I20260812 18:59:21.802392 6310 raft_consensus.cc:2454] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5f9235f90144386bc27513990c1cf8f in term 0.
I20260812 18:59:21.802855 6074 leader_election.cc:302] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 17baa51a90904d65b6a9a07958887c47, 99f725af76044b9db2a594397d1158ee, b5f9235f90144386bc27513990c1cf8f; no voters:
I20260812 18:59:21.803081 6495 raft_consensus.cc:2781] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:59:21.803151 6495 raft_consensus.cc:482] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:59:21.803206 6495 raft_consensus.cc:3037] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:21.804226 6495 raft_consensus.cc:504] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } }
I20260812 18:59:21.804584 6495 leader_election.cc:288] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [CANDIDATE]: Term 1 election: Requested vote from peers 17baa51a90904d65b6a9a07958887c47 (127.29.112.193:41303), 99f725af76044b9db2a594397d1158ee (127.29.112.195:39415)
I20260812 18:59:21.804931 5997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "924fc38e11d747609a75c6adce3053c4" candidate_uuid: "b5f9235f90144386bc27513990c1cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17baa51a90904d65b6a9a07958887c47"
I20260812 18:59:21.805096 5997 raft_consensus.cc:3037] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:21.805931 6310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "924fc38e11d747609a75c6adce3053c4" candidate_uuid: "b5f9235f90144386bc27513990c1cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99f725af76044b9db2a594397d1158ee"
I20260812 18:59:21.806039 6310 raft_consensus.cc:3037] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:21.806227 5997 raft_consensus.cc:2454] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f9235f90144386bc27513990c1cf8f in term 1.
I20260812 18:59:21.806566 6074 leader_election.cc:302] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17baa51a90904d65b6a9a07958887c47, b5f9235f90144386bc27513990c1cf8f; no voters:
I20260812 18:59:21.806737 6495 raft_consensus.cc:2781] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:21.807034 6495 raft_consensus.cc:686] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 1 LEADER]: Becoming Leader. State: Replica: b5f9235f90144386bc27513990c1cf8f, State: Running, Role: LEADER
I20260812 18:59:21.807215 6495 consensus_queue.cc:237] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } }
I20260812 18:59:21.807272 6310 raft_consensus.cc:2454] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5f9235f90144386bc27513990c1cf8f in term 1.
I20260812 18:59:21.807833 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.807496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.194:48137)
0812 18:59:21.807671 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.807739 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260812 18:59:21.809110 5637 catalog_manager.cc:5495] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f reported cstate change: term changed from 0 to 1, leader changed from <none> to b5f9235f90144386bc27513990c1cf8f (127.29.112.194). New cstate: current_term: 1 leader_uuid: "b5f9235f90144386bc27513990c1cf8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f9235f90144386bc27513990c1cf8f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 } health_report { overall_health: UNKNOWN } } }
I20260812 18:59:21.822865 6483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.819919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59874 (local address 127.29.112.194:39297)
0812 18:59:21.820044 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:21.820054 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:21.820074 (+ 20us) server_negotiation.cc:407] Connection header received
0812 18:59:21.820213 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:21.820215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:21.820256 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:21.820318 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:59:21.820546 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.821032 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.821944 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:21.822154 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:21.822226 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:21.822518 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:59:21.822576 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:59:21.822625 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:21.822714 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 18:59:21.822753 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 18:59:21.824837 6310 raft_consensus.cc:1261] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee [term 1 FOLLOWER]: Refusing update from remote peer b5f9235f90144386bc27513990c1cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:21.824860 5997 raft_consensus.cc:1261] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47 [term 1 FOLLOWER]: Refusing update from remote peer b5f9235f90144386bc27513990c1cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:21.825330 6495 consensus_queue.cc:1035] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99f725af76044b9db2a594397d1158ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:21.825477 6497 consensus_queue.cc:1035] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "17baa51a90904d65b6a9a07958887c47" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 41303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:21.827565 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.827257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.195:42431)
0812 18:59:21.827428 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.827493 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 18:59:21.827565 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.827081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.193:59311)
0812 18:59:21.827429 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.827492 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272}
I20260812 18:59:21.830302 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.830031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.194:45001)
0812 18:59:21.830165 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.830218 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 18:59:21.830814 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.830483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.195:43321)
0812 18:59:21.830687 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.830744 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260812 18:59:21.831563 6051 tablet.cc:2369] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:59:21.831781 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.831467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.194:50555)
0812 18:59:21.831634 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.831711 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 18:59:21.831849 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.831164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:40813 (local address 127.29.112.193:45221)
0812 18:59:21.831724 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.831778 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:40813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470}
I20260812 18:59:21.837649 5637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:21.837787 5637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:21.841467 5637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1308435398344058b8a7435586e31d70: Sending DeleteTablet for 3 replicas of tablet 924fc38e11d747609a75c6adce3053c4
I20260812 18:59:21.842176 6131 tablet_service.cc:1514] Processing DeleteTablet for tablet 924fc38e11d747609a75c6adce3053c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:21 UTC) from {username='oryx', principal='oryx/127.29.112.253@KRBTEST.COM'} at 127.0.0.1:59868
I20260812 18:59:21.842178 5975 tablet_service.cc:1514] Processing DeleteTablet for tablet 924fc38e11d747609a75c6adce3053c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:21 UTC) from {username='oryx', principal='oryx/127.29.112.253@KRBTEST.COM'} at 127.0.0.1:48932
I20260812 18:59:21.842245 6287 tablet_service.cc:1514] Processing DeleteTablet for tablet 924fc38e11d747609a75c6adce3053c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:21 UTC) from {username='oryx', principal='oryx/127.29.112.253@KRBTEST.COM'} at 127.0.0.1:34204
I20260812 18:59:21.842512 6512 tablet_replica.cc:330] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f: stopping tablet replica
I20260812 18:59:21.842674 6512 raft_consensus.cc:2229] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:59:21.842695 6514 tablet_replica.cc:330] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee: stopping tablet replica
I20260812 18:59:21.842821 6514 raft_consensus.cc:2229] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:21.842872 6512 raft_consensus.cc:2258] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:21.842934 6514 raft_consensus.cc:2258] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:21.843102 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 5805
I20260812 18:59:21.843142 6513 tablet_replica.cc:330] T 924fc38e11d747609a75c6adce3053c4 P 17baa51a90904d65b6a9a07958887c47: stopping tablet replica
I20260812 18:59:21.843643 6512 ts_tablet_manager.cc:1905] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:21.846071 6512 ts_tablet_manager.cc:1918] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:21.846161 6512 log.cc:1196] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/924fc38e11d747609a75c6adce3053c4
I20260812 18:59:21.846429 6512 ts_tablet_manager.cc:1939] T 924fc38e11d747609a75c6adce3053c4 P b5f9235f90144386bc27513990c1cf8f: Deleting consensus metadata
I20260812 18:59:21.847136 5622 catalog_manager.cc:4841] TS b5f9235f90144386bc27513990c1cf8f (127.29.112.194:39297): tablet 924fc38e11d747609a75c6adce3053c4 (table test-table [id=ba37910063cb4f56ac22f99e19e07d70]) successfully deleted
I20260812 18:59:21.848003 6514 ts_tablet_manager.cc:1905] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:21.850020 6514 ts_tablet_manager.cc:1918] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:21.850159 6514 log.cc:1196] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/924fc38e11d747609a75c6adce3053c4
I20260812 18:59:21.850449 6514 ts_tablet_manager.cc:1939] T 924fc38e11d747609a75c6adce3053c4 P 99f725af76044b9db2a594397d1158ee: Deleting consensus metadata
I20260812 18:59:21.851073 5626 catalog_manager.cc:4841] TS 99f725af76044b9db2a594397d1158ee (127.29.112.195:39415): tablet 924fc38e11d747609a75c6adce3053c4 (table test-table [id=ba37910063cb4f56ac22f99e19e07d70]) successfully deleted
I20260812 18:59:21.855369 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 6056
I20260812 18:59:21.855564 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:21.855300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:41303 (local address 127.0.0.1:48934)
0812 18:59:21.855410 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:21.855463 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:41303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 18:59:21.858047 5624 catalog_manager.cc:4584] TS 17baa51a90904d65b6a9a07958887c47 (127.29.112.193:41303): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 924fc38e11d747609a75c6adce3053c4: Network error: Client connection negotiation failed: client connection to 127.29.112.193:41303: connect: Connection refused (error 111)
I20260812 18:59:21.866441 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 6213
I20260812 18:59:21.876151 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 5485
I20260812 18:59:21.883550 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 5607
2026-08-12T18:59:21Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7314 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:59:21 dist-test-slave-6nmv krb5kdc[6517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:59:21 dist-test-slave-6nmv krb5kdc[6517](info): set up 2 sockets
Aug 12 18:59:21 dist-test-slave-6nmv krb5kdc[6517](info): commencing operation
krb5kdc: starting...
W20260812 18:59:22.995918 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:23 dist-test-slave-6nmv krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.254.keytab.
Entry for principal oryx/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.254.keytab.
I20260812 18:59:23.116140 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:44061
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.254.keytab
--principal=oryx/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:44061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:23.224807 6530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:23.225183 6530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:23.225277 6530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:23.232311 6530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:23.232410 6530 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 18:59:23.232443 6530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:23.232499 6530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:23.232584 6530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:23.232625 6530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:23.242182 6530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:44061
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.254.keytab
--principal=oryx/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:44061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:23.243577 6530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:23.245050 6530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:23.255750 6535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:23.255941 6530 server_base.cc:1034] running on GCE node
W20260812 18:59:23.255820 6536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:23.256238 6538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:23.256866 6530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:23.258019 6530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:23.259193 6530 hybrid_clock.cc:648] HybridClock initialized: now 1786561163259166 us; error 46 us; skew 500 ppm
Aug 12 18:59:23 dist-test-slave-6nmv krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561163, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:23.261978 6530 init.cc:377] Logged in from keytab as oryx/127.29.112.254@KRBTEST.COM (short username oryx)
I20260812 18:59:23.263007 6530 webserver.cc:460] Webserver started at http://127.29.112.254:39395/ using document root <none> and password file <none>
I20260812 18:59:23.263640 6530 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:23.263713 6530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:23.263918 6530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:23.265739 6530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "bd3a7c5697d74e42bfa309b3d254e1c2"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "460f4d6033932204c0fef68719821e5b"
server_key_iv: "7b567d40dcf2a29aea6f158b21b1817f"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.266367 6530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "bd3a7c5697d74e42bfa309b3d254e1c2"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "460f4d6033932204c0fef68719821e5b"
server_key_iv: "7b567d40dcf2a29aea6f158b21b1817f"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.270744 6530 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 18:59:23.272971 6546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:23.274251 6530 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:23.274389 6530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "bd3a7c5697d74e42bfa309b3d254e1c2"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "460f4d6033932204c0fef68719821e5b"
server_key_iv: "7b567d40dcf2a29aea6f158b21b1817f"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.274510 6530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:23.294094 6530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:23.297101 6530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:23.297304 6530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:23.305245 6530 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:44061
I20260812 18:59:23.305267 6627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:44061 every 8 connection(s)
I20260812 18:59:23.306174 6530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:23.308840 6628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:23.312031 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 6530
I20260812 18:59:23.312187 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:23.312461 30147 external_mini_cluster.cc:1428] Setting key 6c25674a19b9082eead4dcad33a83471
I20260812 18:59:23.314697 6628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2: Bootstrap starting.
I20260812 18:59:23.317200 6628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:23.318065 6628 log.cc:824] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:23.320043 6628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2: No bootstrap required, opened a new log
Aug 12 18:59:23 dist-test-slave-6nmv krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:23.324450 6628 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd3a7c5697d74e42bfa309b3d254e1c2" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44061 } }
I20260812 18:59:23.324723 6628 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:23.324782 6628 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd3a7c5697d74e42bfa309b3d254e1c2, State: Initialized, Role: FOLLOWER
I20260812 18:59:23.325206 6628 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd3a7c5697d74e42bfa309b3d254e1c2" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44061 } }
I20260812 18:59:23.325348 6628 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:23.325416 6628 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:23.325511 6628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:23.326495 6628 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd3a7c5697d74e42bfa309b3d254e1c2" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44061 } }
I20260812 18:59:23.326861 6628 leader_election.cc:302] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd3a7c5697d74e42bfa309b3d254e1c2; no voters:
I20260812 18:59:23.327163 6628 leader_election.cc:288] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:23.327307 6634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:23.327548 6634 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [term 1 LEADER]: Becoming Leader. State: Replica: bd3a7c5697d74e42bfa309b3d254e1c2, State: Running, Role: LEADER
I20260812 18:59:23.327678 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:23.313885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36472 (local address 127.29.112.254:44061)
0812 18:59:23.314294 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:23.314312 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:23.314353 (+ 41us) server_negotiation.cc:407] Connection header received
0812 18:59:23.315107 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:23.315129 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:23.315489 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:23.315825 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:23.316405 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.317954 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.318929 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.319525 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.319667 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:23.322643 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:23.322648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:23.322663 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:23.322692 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:23.324610 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:23.325143 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:23.325147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:23.325152 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:23.325278 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:23.325542 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:23.325545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:23.325546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:23.325997 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:23.326196 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:23.326409 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 18:59:23.326728 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":101,"threads_started":1}
I20260812 18:59:23.328320 6634 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd3a7c5697d74e42bfa309b3d254e1c2" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44061 } }
I20260812 18:59:23.328351 6628 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:23.330287 6636 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd3a7c5697d74e42bfa309b3d254e1c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd3a7c5697d74e42bfa309b3d254e1c2" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44061 } } }
I20260812 18:59:23.330382 6636 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:23.330685 6635 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd3a7c5697d74e42bfa309b3d254e1c2. Latest consensus state: current_term: 1 leader_uuid: "bd3a7c5697d74e42bfa309b3d254e1c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd3a7c5697d74e42bfa309b3d254e1c2" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44061 } } }
I20260812 18:59:23.330962 6635 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2 [sys.catalog]: This master's current role is: LEADER
W20260812 18:59:23.333092 6651 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 18:59:23.333204 6651 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 18:59:23.333350 6652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:23.334208 6652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:23.339596 6652 catalog_manager.cc:1347] Generated new cluster ID: 2c2b59f823804f2f83a62e4aa2bda25f
I20260812 18:59:23.339706 6652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:23.355983 6652 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:23.356884 6652 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:23.361091 6652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2: Generated new TSK 0
I20260812 18:59:23.361718 6652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab.
Entry for principal oryx/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab.
I20260812 18:59:23.398496 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44061
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab
--principal=oryx/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:23.520387 6658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:23.520632 6658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:23.520674 6658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:23.524613 6658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:23.524689 6658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:23.524727 6658 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 18:59:23.524824 6658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:23.530256 6658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab
--principal=oryx/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.254:44061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:23.531213 6658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:23.532181 6658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:23.540792 6664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:23.540789 6663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:23.540853 6666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:23.540885 6658 server_base.cc:1034] running on GCE node
I20260812 18:59:23.541363 6658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:23.541919 6658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:23.543067 6658 hybrid_clock.cc:648] HybridClock initialized: now 1786561163543051 us; error 25 us; skew 500 ppm
Aug 12 18:59:23 dist-test-slave-6nmv krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561163, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:23.545647 6658 init.cc:377] Logged in from keytab as oryx/127.29.112.193@KRBTEST.COM (short username oryx)
I20260812 18:59:23.546418 6658 webserver.cc:460] Webserver started at http://127.29.112.193:38213/ using document root <none> and password file <none>
I20260812 18:59:23.546931 6658 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:23.546989 6658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:23.547199 6658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:23.549132 6658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "6f47983fbe1b46ed9036354e0ed776b6"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "f3b9895bf53f48c5f66bf449d7668e70"
server_key_iv: "4fc7cd8f9ac4adde6bd37e556498ae28"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.549609 6658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "6f47983fbe1b46ed9036354e0ed776b6"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "f3b9895bf53f48c5f66bf449d7668e70"
server_key_iv: "4fc7cd8f9ac4adde6bd37e556498ae28"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.552955 6658 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 18:59:23.555284 6675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:23.556425 6658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:23.556540 6658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "6f47983fbe1b46ed9036354e0ed776b6"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "f3b9895bf53f48c5f66bf449d7668e70"
server_key_iv: "4fc7cd8f9ac4adde6bd37e556498ae28"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.556653 6658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:23.575115 6658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:23.578315 6658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:23.578521 6658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:23.579823 6658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:23.579870 6658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:23.579938 6658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:23.579973 6658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 18:59:23 dist-test-slave-6nmv krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561163, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:23.589804 6658 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:45013
I20260812 18:59:23.589874 6815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:45013 every 8 connection(s)
I20260812 18:59:23.590723 6658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:23.593422 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:23.580662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:52195 (local address 127.29.112.254:44061)
0812 18:59:23.580785 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:23.580790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:23.581720 (+ 930us) server_negotiation.cc:407] Connection header received
0812 18:59:23.582823 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:23.582827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:23.582882 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:23.582939 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:23.584494 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.585272 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.586930 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.587220 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.587340 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:23.590025 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:23.590028 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:23.590031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:23.590062 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:23.591459 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:23.592004 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:23.592008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:23.592010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:23.592094 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:23.592422 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:23.592425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:23.592427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:23.592609 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:23.592715 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:23.593158 (+ 443us) server_negotiation.cc:299] Negotiation successful
0812 18:59:23.593274 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 18:59:23.594075 6685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:23.581140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44061 (local address 127.29.112.193:52195)
0812 18:59:23.581577 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:23.581615 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:23.582584 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:23.583306 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:23.583314 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:23.583709 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:23.584318 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:23.584337 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.585426 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.585435 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:23.586795 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:23.586806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.587375 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.587379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:23.587558 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:23.588149 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:23.588170 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:23.589841 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:23.591594 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:23.591600 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:23.591621 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:23.591896 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:23.592182 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:23.592186 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:23.592188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:23.592321 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:23.592713 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:23.592719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:23.592948 (+ 229us) client_negotiation.cc:769] Sending connection context
0812 18:59:23.593105 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 18:59:23.593361 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":121,"threads_started":1}
I20260812 18:59:23.595069 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 6658
I20260812 18:59:23.595227 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:23.595229 6816 heartbeater.cc:347] Connected to a master server at 127.29.112.254:44061
I20260812 18:59:23.595391 6816 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:23.595569 30147 external_mini_cluster.cc:1428] Setting key d993a371df1562efdc41de63fd4ca45a
I20260812 18:59:23.595870 6816 heartbeater.cc:511] Master 127.29.112.254:44061 requested a full tablet report, sending...
I20260812 18:59:23.597379 6569 ts_manager.cc:194] Registered new tserver with Master: 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013)
I20260812 18:59:23.598770 6569 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:52195
I20260812 18:59:23.606829 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:23.598679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:38691 (local address 127.29.112.254:44061)
0812 18:59:23.598822 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:23.598828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:23.599703 (+ 875us) server_negotiation.cc:407] Connection header received
0812 18:59:23.599802 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:23.599805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:23.599845 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:23.599932 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:23.600615 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.601269 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.602085 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.602459 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.602581 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:23.603822 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:23.603825 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:23.603827 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:23.603847 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:23.605217 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:23.605705 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:23.605707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:23.605708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:23.605774 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:23.606080 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:23.606083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:23.606084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:23.606248 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:23.606320 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:23.606639 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 18:59:23.606725 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 18:59:23.606983 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:23.598117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44061 (local address 127.29.112.193:38691)
0812 18:59:23.599601 (+ 1484us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:23.599617 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:23.599728 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:23.600106 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:23.600122 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:23.600351 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:23.600481 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:23.600490 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.601387 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.601389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:23.601955 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:23.601963 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.602585 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.602588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:23.602672 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:23.603188 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:23.603207 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:23.603688 (+ 481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:23.605332 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:23.605335 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:23.605337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:23.605605 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:23.605883 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:23.605889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:23.605891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:23.605978 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:23.606346 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:23.606350 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:23.606455 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 18:59:23.606585 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 18:59:23.606769 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1392,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for oryx/127.29.112.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab.
Entry for principal oryx/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab.
I20260812 18:59:23.624251 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44061
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab
--principal=oryx/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:23.733640 6825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:23.733935 6825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:23.734006 6825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:23.737949 6825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:23.738003 6825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:23.738042 6825 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 18:59:23.738107 6825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:23.743321 6825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab
--principal=oryx/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.254:44061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:23.744393 6825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:23.745445 6825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:23.755164 6836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:23.755210 6832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:23.755194 6830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:23.755462 6825 server_base.cc:1034] running on GCE node
I20260812 18:59:23.755851 6825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:23.756453 6825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:23.757629 6825 hybrid_clock.cc:648] HybridClock initialized: now 1786561163757627 us; error 44 us; skew 500 ppm
Aug 12 18:59:23 dist-test-slave-6nmv krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561163, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:23.760317 6825 init.cc:377] Logged in from keytab as oryx/127.29.112.194@KRBTEST.COM (short username oryx)
I20260812 18:59:23.761159 6825 webserver.cc:460] Webserver started at http://127.29.112.194:43401/ using document root <none> and password file <none>
I20260812 18:59:23.761677 6825 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:23.761735 6825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:23.761960 6825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:23.764027 6825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "412285c987324f11bebdb6a60cb46479"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "a2789a447ac63a465bc79458b3b20ccf"
server_key_iv: "70c6d0b646458823318d9d22c51797f6"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.764582 6825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "412285c987324f11bebdb6a60cb46479"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "a2789a447ac63a465bc79458b3b20ccf"
server_key_iv: "70c6d0b646458823318d9d22c51797f6"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.768623 6825 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 18:59:23.771152 6844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:23.772400 6825 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:23.772504 6825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "412285c987324f11bebdb6a60cb46479"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "a2789a447ac63a465bc79458b3b20ccf"
server_key_iv: "70c6d0b646458823318d9d22c51797f6"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.772645 6825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:23.784159 6825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:23.787058 6825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:23.787278 6825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:23.788772 6825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:23.788825 6825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:23.788894 6825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:23.788933 6825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 18:59:23 dist-test-slave-6nmv krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561163, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:23.801157 6825 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:43271
I20260812 18:59:23.801239 6975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:43271 every 8 connection(s)
I20260812 18:59:23.802181 6825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:23.803612 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:23.789638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:48587 (local address 127.29.112.254:44061)
0812 18:59:23.789765 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:23.789769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:23.790809 (+ 1040us) server_negotiation.cc:407] Connection header received
0812 18:59:23.791755 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:23.791758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:23.791827 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:23.791927 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:23.793457 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.794217 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.795864 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.796166 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.796288 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:23.799659 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:23.799663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:23.799666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:23.799700 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:23.801410 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:23.801961 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:23.801964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:23.801966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:23.802071 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:23.802472 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:23.802475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:23.802478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:23.802730 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:23.802824 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:23.803370 (+ 546us) server_negotiation.cc:299] Negotiation successful
0812 18:59:23.803487 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 18:59:23.804344 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:23.790199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44061 (local address 127.29.112.194:48587)
0812 18:59:23.790668 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:23.790708 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:23.791519 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:23.792105 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:23.792114 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:23.792617 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:23.793309 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:23.793325 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.794348 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.794351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:23.795753 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:23.795766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.796329 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.796333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:23.796523 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:23.797167 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:23.797187 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:23.799477 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:23.801534 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:23.801540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:23.801555 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:23.801865 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:23.802163 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:23.802167 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:23.802169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:23.802371 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:23.802876 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:23.802882 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:23.803131 (+ 249us) client_negotiation.cc:769] Sending connection context
0812 18:59:23.803276 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 18:59:23.803520 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":141,"threads_started":1}
I20260812 18:59:23.805470 6976 heartbeater.cc:347] Connected to a master server at 127.29.112.254:44061
I20260812 18:59:23.805642 6976 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:23.806205 6976 heartbeater.cc:511] Master 127.29.112.254:44061 requested a full tablet report, sending...
I20260812 18:59:23.807258 6567 ts_manager.cc:194] Registered new tserver with Master: 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271)
I20260812 18:59:23.808073 6567 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:48587
I20260812 18:59:23.810669 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 6825
I20260812 18:59:23.810827 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:23.811183 30147 external_mini_cluster.cc:1428] Setting key 8852b06e50ec106c71edbe72999826e5
I20260812 18:59:23.815208 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:23.807724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:45101 (local address 127.29.112.254:44061)
0812 18:59:23.807869 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:23.807874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:23.808031 (+ 157us) server_negotiation.cc:407] Connection header received
0812 18:59:23.808133 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:23.808135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:23.808203 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:23.808272 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:23.808730 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.809215 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.810294 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.810518 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.810597 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:23.811855 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:23.811859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:23.811861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:23.811913 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:23.813473 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:23.813925 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:23.813931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:23.813936 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:23.814028 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:23.814287 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:23.814293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:23.814297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:23.814609 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:23.814713 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:23.814977 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 18:59:23.815100 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 18:59:23.815313 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:23.807663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44061 (local address 127.29.112.194:45101)
0812 18:59:23.807937 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:23.807952 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:23.808052 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:23.808297 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:23.808299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:23.808517 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:23.808612 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:23.808617 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.809341 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.809345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:23.810195 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:23.810203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:23.810646 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:23.810650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:23.810757 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:23.811273 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:23.811290 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:23.811720 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:23.813579 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:23.813581 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:23.813583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:23.813770 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:23.814119 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:23.814121 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:23.814123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:23.814200 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:23.814702 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:23.814704 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:23.814776 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 18:59:23.814896 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 18:59:23.815079 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for oryx/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab.
Entry for principal oryx/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab.
I20260812 18:59:23.840883 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:44061
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab
--principal=oryx/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:23.953279 6991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:23.953522 6991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:23.953588 6991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:23.957517 6991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:23.957582 6991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:23.957667 6991 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 18:59:23.957762 6991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:23.963119 6991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab
--principal=oryx/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.254:44061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:23.964156 6991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:23.965166 6991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:23.974331 6996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:23.974397 6991 server_base.cc:1034] running on GCE node
W20260812 18:59:23.974330 6999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:23.974327 6997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:23.975035 6991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:23.975560 6991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:23.976745 6991 hybrid_clock.cc:648] HybridClock initialized: now 1786561163976717 us; error 38 us; skew 500 ppm
Aug 12 18:59:23 dist-test-slave-6nmv krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561163, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:23.979727 6991 init.cc:377] Logged in from keytab as oryx/127.29.112.195@KRBTEST.COM (short username oryx)
I20260812 18:59:23.980456 6991 webserver.cc:460] Webserver started at http://127.29.112.195:33901/ using document root <none> and password file <none>
I20260812 18:59:23.980952 6991 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:23.981003 6991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:23.981173 6991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:23.983103 6991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "8802ca5e37c14932aa72e51b6ca84122"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "c725701fe2ed07ee3a5563e109e47de0"
server_key_iv: "b282a6eb01aa4e2b3116d5c0c1e58f11"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.983644 6991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "8802ca5e37c14932aa72e51b6ca84122"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "c725701fe2ed07ee3a5563e109e47de0"
server_key_iv: "b282a6eb01aa4e2b3116d5c0c1e58f11"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.987092 6991 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 18:59:23.989303 7008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:23.990446 6991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:23.990569 6991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "8802ca5e37c14932aa72e51b6ca84122"
format_stamp: "Formatted at 2026-08-12 18:59:23 on dist-test-slave-6nmv"
server_key: "c725701fe2ed07ee3a5563e109e47de0"
server_key_iv: "b282a6eb01aa4e2b3116d5c0c1e58f11"
server_key_version: "encryptionkey@0"
I20260812 18:59:23.990662 6991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:24.002413 6991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:24.005564 6991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:24.005757 6991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:24.007300 6991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:24.007370 6991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:24.007440 6991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:24.007494 6991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 18:59:24 dist-test-slave-6nmv krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561163, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:24.018563 6991 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:42069
I20260812 18:59:24.018592 7150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:42069 every 8 connection(s)
I20260812 18:59:24.019680 6991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:59:24.021237 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.008061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:53951 (local address 127.29.112.254:44061)
0812 18:59:24.008230 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.008236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.009017 (+ 781us) server_negotiation.cc:407] Connection header received
0812 18:59:24.010086 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.010090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.010154 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.010218 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:24.011451 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.012155 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.013413 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.013633 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.013746 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.017083 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:24.017087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:24.017090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:24.017126 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:24.019022 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:24.019613 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:24.019616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:24.019617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:24.019697 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:24.020114 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:24.020118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:24.020120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:24.020354 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:24.020468 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.020986 (+ 518us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.021102 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 18:59:24.022116 7019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.008421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44061 (local address 127.29.112.195:53951)
0812 18:59:24.008839 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.008883 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.009838 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.010550 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.010556 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.010848 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:24.011294 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.011306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.012299 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.012302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.013275 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.013285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.013783 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.013787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.014029 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.014630 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:24.014652 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:24.016867 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:24.019142 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:24.019147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:24.019161 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:24.019499 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:24.019817 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:24.019820 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:24.019822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:24.019993 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:24.020525 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:24.020531 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:24.020761 (+ 230us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.020921 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.021178 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":123,"threads_started":1}
I20260812 18:59:24.023386 7151 heartbeater.cc:347] Connected to a master server at 127.29.112.254:44061
I20260812 18:59:24.023523 7151 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:24.024094 7151 heartbeater.cc:511] Master 127.29.112.254:44061 requested a full tablet report, sending...
I20260812 18:59:24.025161 6567 ts_manager.cc:194] Registered new tserver with Master: 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195:42069)
I20260812 18:59:24.025897 6567 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:53951
I20260812 18:59:24.027577 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 6991
I20260812 18:59:24.027736 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:24.028090 30147 external_mini_cluster.cc:1428] Setting key ed0f5a35c8c72dc4107f49cb23ce57ca
I20260812 18:59:24.030189 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 18:59:24.033670 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.025673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:47361 (local address 127.29.112.254:44061)
0812 18:59:24.025799 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.025804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.026021 (+ 217us) server_negotiation.cc:407] Connection header received
0812 18:59:24.026111 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.026114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.026171 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.026248 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:24.026745 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.027510 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.028596 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.028957 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.029052 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.030409 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:24.030413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:24.030416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:24.030439 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:24.032029 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:24.032422 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:24.032428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:24.032433 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:24.032525 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:24.032865 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:24.032870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:24.032875 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:24.033094 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:24.033172 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.033477 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.033564 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 18:59:24.033841 7158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.025597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44061 (local address 127.29.112.195:47361)
0812 18:59:24.025908 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.025925 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.026024 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.026287 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.026290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.026503 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:24.026616 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.026623 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.027665 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.027668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.028470 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.028481 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.029101 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.029105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.029199 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.029686 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:24.029700 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:24.030254 (+ 554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:24.032145 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:24.032147 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:24.032149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:24.032330 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:24.032617 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:24.032619 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:24.032620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:24.032736 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:24.033193 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:24.033197 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:24.033306 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.033424 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.033576 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":108,"threads_started":1}
Aug 12 18:59:24 dist-test-slave-6nmv krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561164, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 18:59:24 dist-test-slave-6nmv krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:24 dist-test-slave-6nmv krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 18:59:24.054975 6631 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:24.049096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.29.112.254:44061)
0812 18:59:24.049222 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.049225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.049236 (+ 11us) server_negotiation.cc:407] Connection header received
0812 18:59:24.049271 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.049274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.049308 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.049372 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:24.050084 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.050573 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.051310 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.051515 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.051625 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.054829 (+ 3204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36482: BlockingRecv error: recv got EOF from 127.0.0.1:36482 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 18:59:24 dist-test-slave-6nmv krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561164, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:24.064291 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.056462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36492 (local address 127.29.112.254:44061)
0812 18:59:24.056588 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.056591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.056710 (+ 119us) server_negotiation.cc:407] Connection header received
0812 18:59:24.056772 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.056774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.056812 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.056894 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:24.057436 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.058005 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.058769 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.058986 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.059106 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.061329 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:24.061332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:24.061347 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:24.061368 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:24.062808 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:24.063213 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:24.063215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:24.063216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:24.063287 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:24.063526 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:24.063528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:24.063529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:24.063687 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:24.063785 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.063981 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.064087 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 18:59:24.067065 6567 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:59:24.070086 6567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:59:24.084930 7184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.078929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:43271 (local address 127.0.0.1:34306)
0812 18:59:24.079929 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.079942 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.080027 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.080702 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.080705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.080731 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:24.080878 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.080885 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.081859 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.081865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.083521 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.083527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.084381 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.084384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.084490 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.084525 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.084637 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.084754 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"mutex_wait_us":225,"spinlock_wait_cycles":3584,"thread_start_us":102,"threads_started":1}
I20260812 18:59:24.085103 7186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.079149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.29.112.195:42069)
0812 18:59:24.079878 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.079889 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.079912 (+ 23us) server_negotiation.cc:407] Connection header received
0812 18:59:24.079990 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.079996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.080229 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.080404 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:24.080688 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.081793 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.083851 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.084562 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.084685 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.084787 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.084857 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.084956 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":102,"threads_started":1}
I20260812 18:59:24.085304 7187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.079318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34306 (local address 127.29.112.194:43271)
0812 18:59:24.079624 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.079637 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.080122 (+ 485us) server_negotiation.cc:407] Connection header received
0812 18:59:24.080200 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.080204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.080397 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.080555 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:24.081031 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.081724 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.083638 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.084254 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.084419 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.084555 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.085079 (+ 524us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.085168 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":114,"threads_started":1}
I20260812 18:59:24.084985 7182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.078539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:42069 (local address 127.0.0.1:32984)
0812 18:59:24.079523 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.079577 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.079710 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.080405 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.080410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.080433 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:24.080568 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.080575 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.081918 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.081920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.083715 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.083727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.084665 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.084669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.084747 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.084766 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.084817 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.084871 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":855,"spinlock_wait_cycles":7296,"thread_start_us":59,"threads_started":1}
I20260812 18:59:24.086408 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.078919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55142 (local address 127.29.112.193:45013)
0812 18:59:24.080017 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.080033 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.080057 (+ 24us) server_negotiation.cc:407] Connection header received
0812 18:59:24.080193 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.080200 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.080398 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.080555 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:24.081009 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.082053 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.083648 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.084432 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.084574 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.084717 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.085294 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.085399 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":101,"threads_started":1}
I20260812 18:59:24.085196 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.078745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45013 (local address 127.0.0.1:55142)
0812 18:59:24.079522 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.079577 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.079710 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.080702 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.080706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.080731 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:24.080880 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.080885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.082152 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.082157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.083505 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.083516 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.084782 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.084786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.084879 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.084900 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.084965 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.085043 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":149,"spinlock_wait_cycles":9728,"thread_start_us":106,"threads_started":1}
I20260812 18:59:24.088454 7070 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccd6559ee154fe98c44fb0900437d75 (DEFAULT_TABLE table=test-table [id=d9c040390bec43a8bd5f464eacf3a27a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:24.088407 6904 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccd6559ee154fe98c44fb0900437d75 (DEFAULT_TABLE table=test-table [id=d9c040390bec43a8bd5f464eacf3a27a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:24.089397 7070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccd6559ee154fe98c44fb0900437d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.089430 6904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccd6559ee154fe98c44fb0900437d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.089141 6732 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccd6559ee154fe98c44fb0900437d75 (DEFAULT_TABLE table=test-table [id=d9c040390bec43a8bd5f464eacf3a27a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:24.090047 6732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccd6559ee154fe98c44fb0900437d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.096112 7188 tablet_bootstrap.cc:492] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: Bootstrap starting.
I20260812 18:59:24.097955 7191 tablet_bootstrap.cc:492] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: Bootstrap starting.
I20260812 18:59:24.098860 7188 tablet_bootstrap.cc:654] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.099529 7188 log.cc:824] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:24.099803 7191 tablet_bootstrap.cc:654] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.100190 7189 tablet_bootstrap.cc:492] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6: Bootstrap starting.
I20260812 18:59:24.100574 7191 log.cc:824] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:24.101550 7188 tablet_bootstrap.cc:492] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: No bootstrap required, opened a new log
I20260812 18:59:24.101737 7188 ts_tablet_manager.cc:1397] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 18:59:24.102087 7189 tablet_bootstrap.cc:654] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.102505 7191 tablet_bootstrap.cc:492] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: No bootstrap required, opened a new log
I20260812 18:59:24.102681 7191 ts_tablet_manager.cc:1397] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 18:59:24.102829 7189 log.cc:824] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:24.104509 7189 tablet_bootstrap.cc:492] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6: No bootstrap required, opened a new log
I20260812 18:59:24.104677 7189 ts_tablet_manager.cc:1397] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 18:59:24.105850 7188 raft_consensus.cc:348] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } }
I20260812 18:59:24.106132 7188 raft_consensus.cc:374] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.106228 7188 raft_consensus.cc:729] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8802ca5e37c14932aa72e51b6ca84122, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.106237 7191 raft_consensus.cc:348] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } }
I20260812 18:59:24.106463 7191 raft_consensus.cc:374] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.106519 7191 raft_consensus.cc:729] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412285c987324f11bebdb6a60cb46479, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.106753 7188 consensus_queue.cc:260] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } }
I20260812 18:59:24.106874 7191 consensus_queue.cc:260] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } }
I20260812 18:59:24.107491 6976 heartbeater.cc:503] Master 127.29.112.254:44061 was elected leader, sending a full tablet report...
I20260812 18:59:24.107733 7188 ts_tablet_manager.cc:1428] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260812 18:59:24.108063 7151 heartbeater.cc:503] Master 127.29.112.254:44061 was elected leader, sending a full tablet report...
I20260812 18:59:24.108144 7189 raft_consensus.cc:348] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } }
I20260812 18:59:24.108354 7189 raft_consensus.cc:374] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.108438 7189 raft_consensus.cc:729] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f47983fbe1b46ed9036354e0ed776b6, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.108790 7191 ts_tablet_manager.cc:1428] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260812 18:59:24.108824 7189 consensus_queue.cc:260] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } }
I20260812 18:59:24.109396 6816 heartbeater.cc:503] Master 127.29.112.254:44061 was elected leader, sending a full tablet report...
I20260812 18:59:24.109716 7189 ts_tablet_manager.cc:1428] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 18:59:24.259341 7197 raft_consensus.cc:482] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:59:24.259537 7197 raft_consensus.cc:504] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } }
I20260812 18:59:24.260650 7197 leader_election.cc:288] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195:42069), 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271)
I20260812 18:59:24.265528 7187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.261048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:50905 (local address 127.29.112.194:43271)
0812 18:59:24.261188 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.261193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.261391 (+ 198us) server_negotiation.cc:407] Connection header received
0812 18:59:24.261498 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.261500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.261541 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.261612 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:24.262031 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.262768 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.264348 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.264899 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.265011 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.265057 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.265351 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.265422 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 18:59:24.265528 7186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.261000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:39365 (local address 127.29.112.195:42069)
0812 18:59:24.261125 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.261129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.261320 (+ 191us) server_negotiation.cc:407] Connection header received
0812 18:59:24.261477 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.261480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.261534 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.261612 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:24.262031 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.262766 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.264348 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.264899 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.265011 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.265057 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.265351 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.265413 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 18:59:24.265636 7198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.260859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:42069 (local address 127.29.112.193:39365)
0812 18:59:24.261217 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.261233 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.261351 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.261635 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.261637 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.261677 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:24.261916 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.261922 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.262870 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.262873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.264235 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.264246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.265014 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.265017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.265159 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.265217 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.265322 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.265480 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":248,"spinlock_wait_cycles":43520,"thread_start_us":105,"threads_started":1}
I20260812 18:59:24.265640 7199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.260976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:43271 (local address 127.29.112.193:50905)
0812 18:59:24.261310 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.261321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.261403 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.261635 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.261637 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.261675 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:24.261915 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.261922 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.262870 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.262873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.264235 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.264246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.265012 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.265015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.265163 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.265218 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.265322 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.265480 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":299,"spinlock_wait_cycles":28416,"thread_start_us":105,"threads_started":1}
I20260812 18:59:24.266374 7091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccd6559ee154fe98c44fb0900437d75" candidate_uuid: "6f47983fbe1b46ed9036354e0ed776b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8802ca5e37c14932aa72e51b6ca84122" is_pre_election: true
I20260812 18:59:24.266414 6927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccd6559ee154fe98c44fb0900437d75" candidate_uuid: "6f47983fbe1b46ed9036354e0ed776b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412285c987324f11bebdb6a60cb46479" is_pre_election: true
I20260812 18:59:24.266666 7091 raft_consensus.cc:2454] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f47983fbe1b46ed9036354e0ed776b6 in term 0.
I20260812 18:59:24.266666 6927 raft_consensus.cc:2454] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f47983fbe1b46ed9036354e0ed776b6 in term 0.
I20260812 18:59:24.267187 6680 leader_election.cc:302] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 412285c987324f11bebdb6a60cb46479, 6f47983fbe1b46ed9036354e0ed776b6, 8802ca5e37c14932aa72e51b6ca84122; no voters:
I20260812 18:59:24.267489 7197 raft_consensus.cc:2781] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:59:24.267568 7197 raft_consensus.cc:482] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:59:24.267599 7197 raft_consensus.cc:3037] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:24.268627 7197 raft_consensus.cc:504] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } }
I20260812 18:59:24.269030 7197 leader_election.cc:288] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [CANDIDATE]: Term 1 election: Requested vote from peers 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195:42069), 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271)
I20260812 18:59:24.269315 7091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccd6559ee154fe98c44fb0900437d75" candidate_uuid: "6f47983fbe1b46ed9036354e0ed776b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8802ca5e37c14932aa72e51b6ca84122"
I20260812 18:59:24.269375 6927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccd6559ee154fe98c44fb0900437d75" candidate_uuid: "6f47983fbe1b46ed9036354e0ed776b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412285c987324f11bebdb6a60cb46479"
I20260812 18:59:24.269433 7091 raft_consensus.cc:3037] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:24.269495 6927 raft_consensus.cc:3037] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Advancing to term 1
W20260812 18:59:24.270730 7152 tablet.cc:2369] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:59:24.270766 6927 raft_consensus.cc:2454] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f47983fbe1b46ed9036354e0ed776b6 in term 1.
I20260812 18:59:24.271067 7091 raft_consensus.cc:2454] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f47983fbe1b46ed9036354e0ed776b6 in term 1.
I20260812 18:59:24.271059 6678 leader_election.cc:302] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412285c987324f11bebdb6a60cb46479, 6f47983fbe1b46ed9036354e0ed776b6; no voters:
I20260812 18:59:24.271270 7197 raft_consensus.cc:2781] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:24.271505 7197 raft_consensus.cc:686] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 LEADER]: Becoming Leader. State: Replica: 6f47983fbe1b46ed9036354e0ed776b6, State: Running, Role: LEADER
I20260812 18:59:24.271837 7197 consensus_queue.cc:237] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } }
I20260812 18:59:24.274763 6567 catalog_manager.cc:5495] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193). New cstate: current_term: 1 leader_uuid: "6f47983fbe1b46ed9036354e0ed776b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } health_report { overall_health: UNKNOWN } } }
W20260812 18:59:24.303328 6978 tablet.cc:2369] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:59:24.320746 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.317439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55148 (local address 127.29.112.193:45013)
0812 18:59:24.317578 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.317581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.317595 (+ 14us) server_negotiation.cc:407] Connection header received
0812 18:59:24.317687 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.317689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.317740 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.317895 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:59:24.318110 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.318614 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.319436 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.319653 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.319742 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.319986 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:59:24.320370 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:59:24.320463 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.320507 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.320557 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 18:59:24.326495 7091 raft_consensus.cc:1261] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Refusing update from remote peer 6f47983fbe1b46ed9036354e0ed776b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:24.326496 6927 raft_consensus.cc:1261] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Refusing update from remote peer 6f47983fbe1b46ed9036354e0ed776b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:24.327265 7200 consensus_queue.cc:1035] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:24.327437 7197 consensus_queue.cc:1035] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:24.336563 7205 mvcc.cc:205] Tried to move back new op lower bound from 7317754529074483200 to 7317754528859848704. Current Snapshot: MvccSnapshot[applied={T|T < 7317754529074483200}]
I20260812 18:59:24.343336 6566 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:24.343524 6566 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:24.345793 6566 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2: Sending DeleteTablet for 3 replicas of tablet 0ccd6559ee154fe98c44fb0900437d75
I20260812 18:59:24.346557 7070 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccd6559ee154fe98c44fb0900437d75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:24 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:32984
I20260812 18:59:24.346632 6732 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccd6559ee154fe98c44fb0900437d75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:24 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:55142
I20260812 18:59:24.347077 7215 tablet_replica.cc:330] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6: stopping tablet replica
I20260812 18:59:24.347227 6904 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccd6559ee154fe98c44fb0900437d75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:24 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:34306
I20260812 18:59:24.347405 7215 raft_consensus.cc:2229] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:59:24.347729 7215 raft_consensus.cc:2258] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:24.347886 7216 tablet_replica.cc:330] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: stopping tablet replica
I20260812 18:59:24.348057 7214 tablet_replica.cc:330] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: stopping tablet replica
I20260812 18:59:24.348006 6566 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 18:59:24.348151 7216 raft_consensus.cc:2229] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:24.348326 7214 raft_consensus.cc:2229] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260812 18:59:24.348383 6566 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:59:24.348412 7216 raft_consensus.cc:2258] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:24.348577 7214 raft_consensus.cc:2258] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:24.349318 7215 ts_tablet_manager.cc:1905] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:24.349920 7214 ts_tablet_manager.cc:1905] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:24.350335 7216 ts_tablet_manager.cc:1905] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:24.352639 7214 ts_tablet_manager.cc:1918] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:24.352778 7214 log.cc:1196] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/0ccd6559ee154fe98c44fb0900437d75
I20260812 18:59:24.353200 7214 ts_tablet_manager.cc:1939] T 0ccd6559ee154fe98c44fb0900437d75 P 8802ca5e37c14932aa72e51b6ca84122: Deleting consensus metadata
I20260812 18:59:24.353267 7216 ts_tablet_manager.cc:1918] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:24.353457 7216 log.cc:1196] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/0ccd6559ee154fe98c44fb0900437d75
I20260812 18:59:24.353860 7216 ts_tablet_manager.cc:1939] T 0ccd6559ee154fe98c44fb0900437d75 P 412285c987324f11bebdb6a60cb46479: Deleting consensus metadata
I20260812 18:59:24.354480 6732 tablet_service.cc:1467] Processing CreateTablet for tablet 0f05507b426f40788a454ba87112ada9 (DEFAULT_TABLE table=test-table [id=773b98d2eecf406fa206a885726d81e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:24.354612 7070 tablet_service.cc:1467] Processing CreateTablet for tablet 0f05507b426f40788a454ba87112ada9 (DEFAULT_TABLE table=test-table [id=773b98d2eecf406fa206a885726d81e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:24.354864 7070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f05507b426f40788a454ba87112ada9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.355094 6550 catalog_manager.cc:4841] TS 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195:42069): tablet 0ccd6559ee154fe98c44fb0900437d75 (table test-table [id=d9c040390bec43a8bd5f464eacf3a27a]) successfully deleted
I20260812 18:59:24.355300 6548 catalog_manager.cc:4841] TS 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271): tablet 0ccd6559ee154fe98c44fb0900437d75 (table test-table [id=d9c040390bec43a8bd5f464eacf3a27a]) successfully deleted
I20260812 18:59:24.355577 6732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f05507b426f40788a454ba87112ada9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.355907 6904 tablet_service.cc:1467] Processing CreateTablet for tablet 0f05507b426f40788a454ba87112ada9 (DEFAULT_TABLE table=test-table [id=773b98d2eecf406fa206a885726d81e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:24.356156 6904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f05507b426f40788a454ba87112ada9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.357759 7189 tablet_bootstrap.cc:492] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6: Bootstrap starting.
I20260812 18:59:24.358326 7191 tablet_bootstrap.cc:492] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479: Bootstrap starting.
I20260812 18:59:24.358440 7188 tablet_bootstrap.cc:492] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122: Bootstrap starting.
I20260812 18:59:24.358767 7189 tablet_bootstrap.cc:654] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.359189 7191 tablet_bootstrap.cc:654] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.359475 7188 tablet_bootstrap.cc:654] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.361029 7188 tablet_bootstrap.cc:492] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122: No bootstrap required, opened a new log
I20260812 18:59:24.361115 7188 ts_tablet_manager.cc:1397] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 18:59:24.361526 7188 raft_consensus.cc:348] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.361618 7188 raft_consensus.cc:374] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.361641 7188 raft_consensus.cc:729] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8802ca5e37c14932aa72e51b6ca84122, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.355578 7215 ts_tablet_manager.cc:1918] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:24.361792 7188 consensus_queue.cc:260] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.361974 7215 log.cc:1196] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/wal/wals/0ccd6559ee154fe98c44fb0900437d75
I20260812 18:59:24.362152 7188 ts_tablet_manager.cc:1428] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 18:59:24.362339 7215 ts_tablet_manager.cc:1939] T 0ccd6559ee154fe98c44fb0900437d75 P 6f47983fbe1b46ed9036354e0ed776b6: Deleting consensus metadata
I20260812 18:59:24.363056 7189 tablet_bootstrap.cc:492] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6: No bootstrap required, opened a new log
I20260812 18:59:24.363138 7189 ts_tablet_manager.cc:1397] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 18:59:24.363517 6547 catalog_manager.cc:4841] TS 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013): tablet 0ccd6559ee154fe98c44fb0900437d75 (table test-table [id=d9c040390bec43a8bd5f464eacf3a27a]) successfully deleted
I20260812 18:59:24.363560 7189 raft_consensus.cc:348] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.363656 7189 raft_consensus.cc:374] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.363684 7189 raft_consensus.cc:729] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f47983fbe1b46ed9036354e0ed776b6, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.363791 7189 consensus_queue.cc:260] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.364020 7189 ts_tablet_manager.cc:1428] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:24.364364 7191 tablet_bootstrap.cc:492] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479: No bootstrap required, opened a new log
I20260812 18:59:24.364528 7191 ts_tablet_manager.cc:1397] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260812 18:59:24.364992 7191 raft_consensus.cc:348] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.365113 7191 raft_consensus.cc:374] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.365144 7191 raft_consensus.cc:729] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412285c987324f11bebdb6a60cb46479, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.365278 7191 consensus_queue.cc:260] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.365566 7191 ts_tablet_manager.cc:1428] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 18:59:24.524930 7196 raft_consensus.cc:482] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:59:24.525094 7196 raft_consensus.cc:504] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.526248 7196 leader_election.cc:288] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013), 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195:42069)
I20260812 18:59:24.531536 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.526646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:40537 (local address 127.29.112.193:45013)
0812 18:59:24.526815 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.526819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.527087 (+ 268us) server_negotiation.cc:407] Connection header received
0812 18:59:24.527448 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.527450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.527492 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.527550 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:24.527821 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.528556 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.530211 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.530865 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.530996 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.531072 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.531336 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.531414 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 18:59:24.531536 7186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.526624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:60067 (local address 127.29.112.195:42069)
0812 18:59:24.526797 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.526804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.527053 (+ 249us) server_negotiation.cc:407] Connection header received
0812 18:59:24.527200 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.527203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.527247 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.527320 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:24.527624 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.528397 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.530214 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.530966 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.531071 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.531120 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.531338 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.531405 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 18:59:24.531657 7221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.526512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:42069 (local address 127.29.112.194:60067)
0812 18:59:24.526960 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.526973 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.527100 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.527346 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.527348 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.527364 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:24.527487 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.527492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.528541 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.528545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.530059 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.530070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.531069 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.531074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.531189 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.531223 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.531305 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.531437 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":160,"spinlock_wait_cycles":19840,"thread_start_us":141,"threads_started":1}
I20260812 18:59:24.531664 7220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.526512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45013 (local address 127.29.112.194:40537)
0812 18:59:24.526920 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.526959 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.527100 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.527593 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.527599 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.527621 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:24.527719 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.527723 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.528663 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.528666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.530061 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.530070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.530988 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.530991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.531177 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.531228 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.531305 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.531442 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":80,"spinlock_wait_cycles":8320,"thread_start_us":129,"threads_started":1}
I20260812 18:59:24.532157 7091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f05507b426f40788a454ba87112ada9" candidate_uuid: "412285c987324f11bebdb6a60cb46479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8802ca5e37c14932aa72e51b6ca84122" is_pre_election: true
I20260812 18:59:24.532315 7091 raft_consensus.cc:2454] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 412285c987324f11bebdb6a60cb46479 in term 0.
I20260812 18:59:24.532279 6741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f05507b426f40788a454ba87112ada9" candidate_uuid: "412285c987324f11bebdb6a60cb46479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f47983fbe1b46ed9036354e0ed776b6" is_pre_election: true
I20260812 18:59:24.532495 6741 raft_consensus.cc:2454] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 412285c987324f11bebdb6a60cb46479 in term 0.
I20260812 18:59:24.532707 6849 leader_election.cc:302] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412285c987324f11bebdb6a60cb46479, 8802ca5e37c14932aa72e51b6ca84122; no voters:
I20260812 18:59:24.533004 7196 raft_consensus.cc:2781] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:59:24.533066 7196 raft_consensus.cc:482] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:59:24.533103 7196 raft_consensus.cc:3037] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:24.534142 7196 raft_consensus.cc:504] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.534498 7196 leader_election.cc:288] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [CANDIDATE]: Term 1 election: Requested vote from peers 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013), 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195:42069)
I20260812 18:59:24.534847 7091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f05507b426f40788a454ba87112ada9" candidate_uuid: "412285c987324f11bebdb6a60cb46479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8802ca5e37c14932aa72e51b6ca84122"
I20260812 18:59:24.534847 6741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f05507b426f40788a454ba87112ada9" candidate_uuid: "412285c987324f11bebdb6a60cb46479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f47983fbe1b46ed9036354e0ed776b6"
I20260812 18:59:24.534965 7091 raft_consensus.cc:3037] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:24.534965 6741 raft_consensus.cc:3037] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:24.536106 6741 raft_consensus.cc:2454] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 412285c987324f11bebdb6a60cb46479 in term 1.
I20260812 18:59:24.536106 7091 raft_consensus.cc:2454] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 412285c987324f11bebdb6a60cb46479 in term 1.
I20260812 18:59:24.536386 6849 leader_election.cc:302] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412285c987324f11bebdb6a60cb46479, 8802ca5e37c14932aa72e51b6ca84122; no voters:
I20260812 18:59:24.536589 7196 raft_consensus.cc:2781] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:24.536821 7196 raft_consensus.cc:686] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 1 LEADER]: Becoming Leader. State: Replica: 412285c987324f11bebdb6a60cb46479, State: Running, Role: LEADER
I20260812 18:59:24.537061 7196 consensus_queue.cc:237] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.539238 6567 catalog_manager.cc:5495] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 reported cstate change: term changed from 0 to 1, leader changed from <none> to 412285c987324f11bebdb6a60cb46479 (127.29.112.194). New cstate: current_term: 1 leader_uuid: "412285c987324f11bebdb6a60cb46479" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } health_report { overall_health: UNKNOWN } } }
Aug 12 18:59:24 dist-test-slave-6nmv krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561163, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.254@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:24.588307 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.579581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.29.112.254:44061)
0812 18:59:24.579864 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.579875 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.580032 (+ 157us) server_negotiation.cc:407] Connection header received
0812 18:59:24.580208 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.580211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.580255 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.580343 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:24.581006 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.581585 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.582445 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.582672 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.582778 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.585108 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:24.585116 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:24.585118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:24.585138 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:24.586552 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:24.587025 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:24.587031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:24.587033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:24.587116 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:24.587388 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:24.587392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:24.587394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:24.587580 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:24.587686 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.587995 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.588160 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":183,"thread_start_us":81,"threads_started":1}
I20260812 18:59:24.588307 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.579515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44061 (local address 127.0.0.1:48030)
0812 18:59:24.579947 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.579959 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.580099 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.580364 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.580367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.580780 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:24.580867 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.580874 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.581691 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.581696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.582317 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.582326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.582780 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.582783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.582861 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.583514 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:24.583543 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:24.584963 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:24.586669 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:24.586678 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:24.586684 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:24.586916 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:24.587206 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:24.587211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:24.587212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:24.587296 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:24.587695 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:24.587704 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:24.587813 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.587946 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.588099 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":119,"threads_started":1}
I20260812 18:59:24.592430 6567 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:48030:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 18:59:24.592702 6817 tablet.cc:2369] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 18:59:24.593245 6567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:59:24.598363 6732 tablet_service.cc:1467] Processing CreateTablet for tablet 50340f5c26834287852453554d00f352 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f344acbbfe2473684e4b8e3d2b1c625]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 18:59:24.598526 6904 tablet_service.cc:1467] Processing CreateTablet for tablet 50340f5c26834287852453554d00f352 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f344acbbfe2473684e4b8e3d2b1c625]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 18:59:24.598731 6732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50340f5c26834287852453554d00f352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.598681 7070 tablet_service.cc:1467] Processing CreateTablet for tablet 50340f5c26834287852453554d00f352 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f344acbbfe2473684e4b8e3d2b1c625]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 18:59:24.598868 6904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50340f5c26834287852453554d00f352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.599128 7070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50340f5c26834287852453554d00f352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.601660 7191 tablet_bootstrap.cc:492] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479: Bootstrap starting.
I20260812 18:59:24.602552 7191 tablet_bootstrap.cc:654] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.603184 7189 tablet_bootstrap.cc:492] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6: Bootstrap starting.
I20260812 18:59:24.603899 7189 tablet_bootstrap.cc:654] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.605717 7189 tablet_bootstrap.cc:492] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6: No bootstrap required, opened a new log
I20260812 18:59:24.605795 7189 ts_tablet_manager.cc:1397] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 18:59:24.606201 7189 raft_consensus.cc:348] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.606312 7189 raft_consensus.cc:374] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.606341 7189 raft_consensus.cc:729] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f47983fbe1b46ed9036354e0ed776b6, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.606451 7189 consensus_queue.cc:260] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.606686 7189 ts_tablet_manager.cc:1428] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:24.606845 7197 tablet_replica.cc:439] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } }
I20260812 18:59:24.606948 7197 tablet_replica.cc:442] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:59:24.607378 7191 tablet_bootstrap.cc:492] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479: No bootstrap required, opened a new log
I20260812 18:59:24.607461 7191 ts_tablet_manager.cc:1397] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 18:59:24.607851 7191 raft_consensus.cc:348] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.607944 7191 raft_consensus.cc:374] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.607971 7191 raft_consensus.cc:729] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412285c987324f11bebdb6a60cb46479, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.608089 7191 consensus_queue.cc:260] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.608315 7191 ts_tablet_manager.cc:1428] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:24.608320 7196 tablet_replica.cc:439] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } }
I20260812 18:59:24.608505 7196 tablet_replica.cc:442] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:59:24.611274 7188 tablet_bootstrap.cc:492] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: Bootstrap starting.
I20260812 18:59:24.612329 7188 tablet_bootstrap.cc:654] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.613479 7188 tablet_bootstrap.cc:492] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: No bootstrap required, opened a new log
I20260812 18:59:24.613601 7188 ts_tablet_manager.cc:1397] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 18:59:24.614045 7188 raft_consensus.cc:348] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.614161 7188 raft_consensus.cc:374] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.614204 7188 raft_consensus.cc:729] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8802ca5e37c14932aa72e51b6ca84122, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.614339 7188 consensus_queue.cc:260] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.614624 7188 ts_tablet_manager.cc:1428] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:24.614609 7195 tablet_replica.cc:439] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } }
I20260812 18:59:24.614746 7195 tablet_replica.cc:442] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:59:24.809324 7195 raft_consensus.cc:482] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:59:24.809480 7195 raft_consensus.cc:504] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.810626 7195 leader_election.cc:288] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013), 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271)
I20260812 18:59:24.815366 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.810872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:43271 (local address 127.29.112.195:56829)
0812 18:59:24.811207 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.811221 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.811315 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.811848 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.811851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.811871 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:24.811992 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.811997 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.812742 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.812747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.814203 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.814214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.814915 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.814930 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.815043 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.815068 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.815134 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.815209 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":50,"thread_start_us":73,"threads_started":1}
I20260812 18:59:24.815382 7238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.811367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:56829 (local address 127.29.112.194:43271)
0812 18:59:24.811621 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.811625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.811639 (+ 14us) server_negotiation.cc:407] Connection header received
0812 18:59:24.811700 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.811703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.811738 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.811821 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:24.812114 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.812622 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.814359 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.814779 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.814938 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.814994 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.815193 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.815266 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260812 18:59:24.815685 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.811105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:56903 (local address 127.29.112.193:45013)
0812 18:59:24.811240 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.811246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.811264 (+ 18us) server_negotiation.cc:407] Connection header received
0812 18:59:24.811456 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.811462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.811532 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.811648 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:24.811949 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.812744 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.814335 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.814978 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.815084 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.815127 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.815451 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.815565 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 18:59:24.815385 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.810857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45013 (local address 127.29.112.195:56903)
0812 18:59:24.811193 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.811208 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.811299 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.811678 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.811681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.811697 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:24.811825 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.811834 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.812849 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.812852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.814207 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.814214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.815070 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.815072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.815165 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.815181 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.815242 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.815289 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":50,"thread_start_us":85,"threads_started":1}
I20260812 18:59:24.816284 6927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50340f5c26834287852453554d00f352" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412285c987324f11bebdb6a60cb46479" is_pre_election: true
I20260812 18:59:24.816435 6927 raft_consensus.cc:2454] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 0.
I20260812 18:59:24.816476 6741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50340f5c26834287852453554d00f352" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f47983fbe1b46ed9036354e0ed776b6" is_pre_election: true
I20260812 18:59:24.816617 6741 raft_consensus.cc:2454] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 0.
I20260812 18:59:24.816823 7010 leader_election.cc:302] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412285c987324f11bebdb6a60cb46479, 8802ca5e37c14932aa72e51b6ca84122; no voters:
I20260812 18:59:24.817108 7195 raft_consensus.cc:2781] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:59:24.817171 7195 raft_consensus.cc:482] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:59:24.817209 7195 raft_consensus.cc:3037] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:24.818347 7195 raft_consensus.cc:504] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.818701 7195 leader_election.cc:288] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 election: Requested vote from peers 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013), 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271)
I20260812 18:59:24.819036 6927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50340f5c26834287852453554d00f352" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412285c987324f11bebdb6a60cb46479"
I20260812 18:59:24.819067 6741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50340f5c26834287852453554d00f352" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f47983fbe1b46ed9036354e0ed776b6"
I20260812 18:59:24.819131 6927 raft_consensus.cc:3037] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:24.819203 6741 raft_consensus.cc:3037] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:24.820145 6741 raft_consensus.cc:2454] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 1.
I20260812 18:59:24.820384 6927 raft_consensus.cc:2454] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 1.
I20260812 18:59:24.820451 7009 leader_election.cc:302] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f47983fbe1b46ed9036354e0ed776b6, 8802ca5e37c14932aa72e51b6ca84122; no voters:
I20260812 18:59:24.820667 7195 raft_consensus.cc:2781] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:24.820827 7195 raft_consensus.cc:686] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 LEADER]: Becoming Leader. State: Replica: 8802ca5e37c14932aa72e51b6ca84122, State: Running, Role: LEADER
I20260812 18:59:24.821081 7195 consensus_queue.cc:237] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.822317 7239 tablet_replica.cc:439] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: TxnStatusTablet state changed. Reason: New leader 8802ca5e37c14932aa72e51b6ca84122. Latest consensus state: current_term: 1 leader_uuid: "8802ca5e37c14932aa72e51b6ca84122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } }
I20260812 18:59:24.822433 7239 tablet_replica.cc:442] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: This TxnStatusTablet replica's current role is: LEADER
I20260812 18:59:24.822700 7241 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 18:59:24.822850 7241 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 18:59:24.823539 6567 catalog_manager.cc:5495] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195). New cstate: current_term: 1 leader_uuid: "8802ca5e37c14932aa72e51b6ca84122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260812 18:59:24.844697 7186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.840952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55860 (local address 127.29.112.195:42069)
0812 18:59:24.841062 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.841065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.841152 (+ 87us) server_negotiation.cc:407] Connection header received
0812 18:59:24.841303 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.841305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.841370 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.841447 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:59:24.841716 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.842250 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.843398 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.843588 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.843656 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.843973 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:59:24.844360 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:59:24.844441 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.844556 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.844600 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 18:59:24.844764 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.840878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:42069 (local address 127.0.0.1:55860)
0812 18:59:24.841086 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.841099 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.841196 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.841486 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.841489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.841497 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:59:24.841605 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.841611 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.842350 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.842353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.843285 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.843295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.843707 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.843710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.843797 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.843849 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:59:24.844456 (+ 607us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:59:24.844471 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.844548 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.844631 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260812 18:59:24.847427 6927 raft_consensus.cc:1261] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Refusing update from remote peer 8802ca5e37c14932aa72e51b6ca84122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:24.847633 6741 raft_consensus.cc:1261] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Refusing update from remote peer 8802ca5e37c14932aa72e51b6ca84122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:24.847900 7239 consensus_queue.cc:1035] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:24.848176 7195 consensus_queue.cc:1035] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:24.849202 7196 tablet_replica.cc:439] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479: TxnStatusTablet state changed. Reason: New leader 8802ca5e37c14932aa72e51b6ca84122. Latest consensus state: current_term: 1 leader_uuid: "8802ca5e37c14932aa72e51b6ca84122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } }
I20260812 18:59:24.849303 7196 tablet_replica.cc:442] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:59:24.850136 7197 tablet_replica.cc:439] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6: TxnStatusTablet state changed. Reason: New leader 8802ca5e37c14932aa72e51b6ca84122. Latest consensus state: current_term: 1 leader_uuid: "8802ca5e37c14932aa72e51b6ca84122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } }
I20260812 18:59:24.850248 7197 tablet_replica.cc:442] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:59:24.852144 7239 tablet_replica.cc:439] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8802ca5e37c14932aa72e51b6ca84122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } }
I20260812 18:59:24.852239 7239 tablet_replica.cc:442] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: This TxnStatusTablet replica's current role is: LEADER
I20260812 18:59:24.852325 7246 mvcc.cc:205] Tried to move back new op lower bound from 7317754531210366976 to 7317754531109281792. Current Snapshot: MvccSnapshot[applied={T|T < 7317754531210366976}]
I20260812 18:59:24.852612 7197 tablet_replica.cc:439] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8802ca5e37c14932aa72e51b6ca84122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } }
I20260812 18:59:24.852720 7197 tablet_replica.cc:442] T 50340f5c26834287852453554d00f352 P 6f47983fbe1b46ed9036354e0ed776b6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:59:24.854514 7249 mvcc.cc:205] Tried to move back new op lower bound from 7317754531210366976 to 7317754531109281792. Current Snapshot: MvccSnapshot[applied={T|T < 7317754531210366976}]
I20260812 18:59:24.855628 7195 tablet_replica.cc:439] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8802ca5e37c14932aa72e51b6ca84122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } }
I20260812 18:59:24.855726 7195 tablet_replica.cc:442] T 50340f5c26834287852453554d00f352 P 8802ca5e37c14932aa72e51b6ca84122: This TxnStatusTablet replica's current role is: LEADER
I20260812 18:59:24.857053 7196 tablet_replica.cc:439] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8802ca5e37c14932aa72e51b6ca84122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } } }
I20260812 18:59:24.857164 7196 tablet_replica.cc:442] T 50340f5c26834287852453554d00f352 P 412285c987324f11bebdb6a60cb46479: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 18:59:24.861937 7238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.857752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.29.112.194:43271)
0812 18:59:24.857880 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.857885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.858110 (+ 225us) server_negotiation.cc:407] Connection header received
0812 18:59:24.858314 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.858318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.858375 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.858447 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:59:24.858899 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.859627 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.860539 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.860841 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.860935 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.861155 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:59:24.861595 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:59:24.861695 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.861758 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.861829 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 18:59:24.871434 7186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.868199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:49233 (local address 127.29.112.195:42069)
0812 18:59:24.868318 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.868323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.868486 (+ 163us) server_negotiation.cc:407] Connection header received
0812 18:59:24.868592 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.868594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.868630 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.868690 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:59:24.868918 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.869348 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.870544 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.870735 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.870810 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.871086 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:59:24.871142 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:59:24.871192 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.871309 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.871348 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 18:59:24.871521 7259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.868130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:42069 (local address 127.29.112.194:49233)
0812 18:59:24.868398 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.868420 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.868514 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.868717 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.868720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.868727 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:59:24.868824 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.868829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.869448 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.869451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.870431 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.870442 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.870835 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.870838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.870920 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.870967 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:59:24.871222 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:59:24.871228 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.871294 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.871367 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":106,"threads_started":1}
I20260812 18:59:24.878432 6741 raft_consensus.cc:1261] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Refusing update from remote peer 412285c987324f11bebdb6a60cb46479: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:24.878611 7091 raft_consensus.cc:1261] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Refusing update from remote peer 412285c987324f11bebdb6a60cb46479: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:24.878909 7222 consensus_queue.cc:1035] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:24.879081 7196 consensus_queue.cc:1035] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:24.901484 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.897790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:43271 (local address 127.29.112.195:49985)
0812 18:59:24.898143 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:24.898155 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:24.898253 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:24.898469 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:24.898471 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:24.898476 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:59:24.898573 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.898577 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.899543 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.899546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.900294 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:24.900304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.900789 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.900792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:24.900886 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.900926 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:59:24.901222 (+ 296us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:59:24.901230 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 18:59:24.901286 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 18:59:24.901355 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":137,"threads_started":1}
I20260812 18:59:24.901469 7238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.897894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:49985 (local address 127.29.112.194:43271)
0812 18:59:24.898020 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.898025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.898243 (+ 218us) server_negotiation.cc:407] Connection header received
0812 18:59:24.898322 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.898325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.898375 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.898466 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:59:24.898678 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.899434 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.900392 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.900677 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.900786 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.901067 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:59:24.901140 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:59:24.901215 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.901303 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.901350 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 18:59:24.924710 6567 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:24.924888 6567 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:24.927366 6567 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2: Sending DeleteTablet for 3 replicas of tablet 0f05507b426f40788a454ba87112ada9
I20260812 18:59:24.928076 6904 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f05507b426f40788a454ba87112ada9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:24 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:34306
I20260812 18:59:24.928167 6732 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f05507b426f40788a454ba87112ada9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:24 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:55142
I20260812 18:59:24.928828 7070 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f05507b426f40788a454ba87112ada9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:24 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:32984
I20260812 18:59:24.929253 7272 tablet_replica.cc:330] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6: stopping tablet replica
I20260812 18:59:24.929427 7272 raft_consensus.cc:2229] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:24.929641 7272 raft_consensus.cc:2258] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:24.930497 7272 ts_tablet_manager.cc:1905] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:24.930729 7273 tablet_replica.cc:330] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122: stopping tablet replica
I20260812 18:59:24.933736 7272 ts_tablet_manager.cc:1918] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:59:24.933965 7273 raft_consensus.cc:2229] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:24.934140 7273 raft_consensus.cc:2258] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:24.934854 7273 ts_tablet_manager.cc:1905] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:24.936460 7271 tablet_replica.cc:330] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479: stopping tablet replica
I20260812 18:59:24.936640 7271 raft_consensus.cc:2229] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:59:24.937023 7271 raft_consensus.cc:2258] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:24.937618 7273 ts_tablet_manager.cc:1918] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:59:24.937731 7271 ts_tablet_manager.cc:1905] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:24.937768 7272 log.cc:1196] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/wal/wals/0f05507b426f40788a454ba87112ada9
I20260812 18:59:24.938081 7272 ts_tablet_manager.cc:1939] T 0f05507b426f40788a454ba87112ada9 P 6f47983fbe1b46ed9036354e0ed776b6: Deleting consensus metadata
I20260812 18:59:24.938956 6547 catalog_manager.cc:4841] TS 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013): tablet 0f05507b426f40788a454ba87112ada9 (table test-table [id=773b98d2eecf406fa206a885726d81e4]) successfully deleted
I20260812 18:59:24.939229 7273 log.cc:1196] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/0f05507b426f40788a454ba87112ada9
I20260812 18:59:24.939591 7273 ts_tablet_manager.cc:1939] T 0f05507b426f40788a454ba87112ada9 P 8802ca5e37c14932aa72e51b6ca84122: Deleting consensus metadata
I20260812 18:59:24.940294 6550 catalog_manager.cc:4841] TS 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195:42069): tablet 0f05507b426f40788a454ba87112ada9 (table test-table [id=773b98d2eecf406fa206a885726d81e4]) successfully deleted
I20260812 18:59:24.942466 7271 ts_tablet_manager.cc:1918] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:59:24.942596 7271 log.cc:1196] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/0f05507b426f40788a454ba87112ada9
I20260812 18:59:24.942901 7271 ts_tablet_manager.cc:1939] T 0f05507b426f40788a454ba87112ada9 P 412285c987324f11bebdb6a60cb46479: Deleting consensus metadata
I20260812 18:59:24.943576 6548 catalog_manager.cc:4841] TS 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271): tablet 0f05507b426f40788a454ba87112ada9 (table test-table [id=773b98d2eecf406fa206a885726d81e4]) successfully deleted
Aug 12 18:59:24 dist-test-slave-6nmv krb5kdc[6517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 18:59:24 dist-test-slave-6nmv krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:24 dist-test-slave-6nmv krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 18:59:24.966085 7231 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:24.958099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48040 (local address 127.29.112.254:44061)
0812 18:59:24.958497 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.958502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.958520 (+ 18us) server_negotiation.cc:407] Connection header received
0812 18:59:24.958626 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.958629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.958686 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.958777 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:24.959694 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.960584 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.961860 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.962182 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.962297 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.965955 (+ 3658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48040: BlockingRecv error: recv got EOF from 127.0.0.1:48040 (error 108)
Metrics: {"server-negotiator.queue_time_us":299}
Aug 12 18:59:24 dist-test-slave-6nmv krb5kdc[6517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM
I20260812 18:59:24.978044 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:24.967703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48054 (local address 127.29.112.254:44061)
0812 18:59:24.967815 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:24.967820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:24.968294 (+ 474us) server_negotiation.cc:407] Connection header received
0812 18:59:24.968382 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:24.968385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:24.968444 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:24.968530 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:24.969169 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.969895 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.970941 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:24.971247 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:24.971370 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:24.974210 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:24.974213 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:24.974216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:24.974241 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:24.975966 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:24.976469 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:24.976473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:24.976475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:24.976567 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:24.976952 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:24.976955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:24.976957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:24.977174 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:24.977303 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:24.977676 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 18:59:24.977855 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 18:59:24.980759 6567 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:59:24.981244 6567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:59:24.986163 6904 tablet_service.cc:1467] Processing CreateTablet for tablet 5c4986c93adb4f97bb75a450364f97f2 (DEFAULT_TABLE table=test-table [id=5a8eecd62401419cbf2b3762c087f314]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:24.986338 7070 tablet_service.cc:1467] Processing CreateTablet for tablet 5c4986c93adb4f97bb75a450364f97f2 (DEFAULT_TABLE table=test-table [id=5a8eecd62401419cbf2b3762c087f314]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:24.986483 6904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c4986c93adb4f97bb75a450364f97f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.986627 7070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c4986c93adb4f97bb75a450364f97f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.988942 7191 tablet_bootstrap.cc:492] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479: Bootstrap starting.
I20260812 18:59:24.988986 7188 tablet_bootstrap.cc:492] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122: Bootstrap starting.
I20260812 18:59:24.989879 7191 tablet_bootstrap.cc:654] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.991758 6732 tablet_service.cc:1467] Processing CreateTablet for tablet 5c4986c93adb4f97bb75a450364f97f2 (DEFAULT_TABLE table=test-table [id=5a8eecd62401419cbf2b3762c087f314]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:24.992054 6732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c4986c93adb4f97bb75a450364f97f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:24.992638 7191 tablet_bootstrap.cc:492] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479: No bootstrap required, opened a new log
I20260812 18:59:24.992712 7191 ts_tablet_manager.cc:1397] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260812 18:59:24.993175 7191 raft_consensus.cc:348] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.993278 7191 raft_consensus.cc:374] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.993307 7191 raft_consensus.cc:729] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412285c987324f11bebdb6a60cb46479, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.993418 7191 consensus_queue.cc:260] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.993664 7191 ts_tablet_manager.cc:1428] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 18:59:24.994522 7189 tablet_bootstrap.cc:492] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6: Bootstrap starting.
I20260812 18:59:24.995303 7189 tablet_bootstrap.cc:654] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.996831 7188 tablet_bootstrap.cc:654] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:24.998314 7189 tablet_bootstrap.cc:492] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6: No bootstrap required, opened a new log
I20260812 18:59:24.998416 7189 ts_tablet_manager.cc:1397] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 18:59:24.998579 7188 tablet_bootstrap.cc:492] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122: No bootstrap required, opened a new log
I20260812 18:59:24.998651 7188 ts_tablet_manager.cc:1397] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260812 18:59:24.998761 7189 raft_consensus.cc:348] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.998858 7189 raft_consensus.cc:374] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.998883 7189 raft_consensus.cc:729] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f47983fbe1b46ed9036354e0ed776b6, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.998977 7188 raft_consensus.cc:348] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.999015 7189 consensus_queue.cc:260] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.999080 7188 raft_consensus.cc:374] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:24.999109 7188 raft_consensus.cc:729] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8802ca5e37c14932aa72e51b6ca84122, State: Initialized, Role: FOLLOWER
I20260812 18:59:24.999241 7188 consensus_queue.cc:260] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:24.999290 7189 ts_tablet_manager.cc:1428] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:24.999496 7188 ts_tablet_manager.cc:1428] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:25.164331 7260 raft_consensus.cc:482] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:59:25.164481 7260 raft_consensus.cc:504] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.164921 7260 leader_election.cc:288] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013), 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271)
I20260812 18:59:25.165402 6741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c4986c93adb4f97bb75a450364f97f2" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f47983fbe1b46ed9036354e0ed776b6" is_pre_election: true
I20260812 18:59:25.165422 6927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c4986c93adb4f97bb75a450364f97f2" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412285c987324f11bebdb6a60cb46479" is_pre_election: true
I20260812 18:59:25.165581 6741 raft_consensus.cc:2454] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 0.
I20260812 18:59:25.165586 6927 raft_consensus.cc:2454] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 0.
I20260812 18:59:25.165930 7010 leader_election.cc:302] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412285c987324f11bebdb6a60cb46479, 8802ca5e37c14932aa72e51b6ca84122; no voters:
I20260812 18:59:25.166159 7260 raft_consensus.cc:2781] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:59:25.166232 7260 raft_consensus.cc:482] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:59:25.166273 7260 raft_consensus.cc:3037] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:25.167359 7260 raft_consensus.cc:504] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.167737 7260 leader_election.cc:288] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 election: Requested vote from peers 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013), 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271)
I20260812 18:59:25.168025 6741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c4986c93adb4f97bb75a450364f97f2" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f47983fbe1b46ed9036354e0ed776b6"
I20260812 18:59:25.168123 6927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c4986c93adb4f97bb75a450364f97f2" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412285c987324f11bebdb6a60cb46479"
I20260812 18:59:25.168196 6741 raft_consensus.cc:3037] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:25.168256 6927 raft_consensus.cc:3037] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:25.169085 6741 raft_consensus.cc:2454] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 1.
I20260812 18:59:25.169263 6927 raft_consensus.cc:2454] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 1.
I20260812 18:59:25.169385 7009 leader_election.cc:302] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f47983fbe1b46ed9036354e0ed776b6, 8802ca5e37c14932aa72e51b6ca84122; no voters:
I20260812 18:59:25.169581 7260 raft_consensus.cc:2781] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:25.169705 7260 raft_consensus.cc:686] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 LEADER]: Becoming Leader. State: Replica: 8802ca5e37c14932aa72e51b6ca84122, State: Running, Role: LEADER
I20260812 18:59:25.169867 7260 consensus_queue.cc:237] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.171473 6569 catalog_manager.cc:5495] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195). New cstate: current_term: 1 leader_uuid: "8802ca5e37c14932aa72e51b6ca84122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260812 18:59:25.186287 7186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:25.183600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55874 (local address 127.29.112.195:42069)
0812 18:59:25.183732 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:25.183735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:25.183747 (+ 12us) server_negotiation.cc:407] Connection header received
0812 18:59:25.183845 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:25.183847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:25.183883 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:25.183941 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:59:25.184174 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:25.184668 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:25.185413 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:25.185598 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:25.185665 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:25.185930 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:59:25.185987 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:59:25.186040 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:25.186116 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 18:59:25.186149 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 18:59:25.188374 6927 raft_consensus.cc:1261] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Refusing update from remote peer 8802ca5e37c14932aa72e51b6ca84122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:25.188388 6741 raft_consensus.cc:1261] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Refusing update from remote peer 8802ca5e37c14932aa72e51b6ca84122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:25.188792 7260 consensus_queue.cc:1035] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:25.188956 7195 consensus_queue.cc:1035] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:25.198942 6569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:25.199102 6569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:25.201051 6569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2: Sending DeleteTablet for 3 replicas of tablet 5c4986c93adb4f97bb75a450364f97f2
I20260812 18:59:25.201606 6904 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c4986c93adb4f97bb75a450364f97f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:25 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:34306
I20260812 18:59:25.201606 6732 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c4986c93adb4f97bb75a450364f97f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:25 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:55142
I20260812 18:59:25.201716 7070 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c4986c93adb4f97bb75a450364f97f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:25 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:32984
I20260812 18:59:25.201774 7272 tablet_replica.cc:330] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6: stopping tablet replica
I20260812 18:59:25.201774 7271 tablet_replica.cc:330] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479: stopping tablet replica
I20260812 18:59:25.201895 7273 tablet_replica.cc:330] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122: stopping tablet replica
I20260812 18:59:25.201895 7272 raft_consensus.cc:2229] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:25.201902 7271 raft_consensus.cc:2229] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:25.202008 7273 raft_consensus.cc:2229] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:59:25.202006 7272 raft_consensus.cc:2258] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:25.202019 7271 raft_consensus.cc:2258] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:25.202256 7273 raft_consensus.cc:2258] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:25.202668 7272 ts_tablet_manager.cc:1905] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:25.202736 7271 ts_tablet_manager.cc:1905] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:25.203019 7273 ts_tablet_manager.cc:1905] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:25.204983 7272 ts_tablet_manager.cc:1918] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:25.204983 7271 ts_tablet_manager.cc:1918] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:25.205052 7272 log.cc:1196] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-0/wal/wals/5c4986c93adb4f97bb75a450364f97f2
I20260812 18:59:25.205057 7271 log.cc:1196] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/5c4986c93adb4f97bb75a450364f97f2
I20260812 18:59:25.205354 7272 ts_tablet_manager.cc:1939] T 5c4986c93adb4f97bb75a450364f97f2 P 6f47983fbe1b46ed9036354e0ed776b6: Deleting consensus metadata
I20260812 18:59:25.205384 7273 ts_tablet_manager.cc:1918] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:25.205430 7271 ts_tablet_manager.cc:1939] T 5c4986c93adb4f97bb75a450364f97f2 P 412285c987324f11bebdb6a60cb46479: Deleting consensus metadata
I20260812 18:59:25.205454 7273 log.cc:1196] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/5c4986c93adb4f97bb75a450364f97f2
I20260812 18:59:25.205770 7273 ts_tablet_manager.cc:1939] T 5c4986c93adb4f97bb75a450364f97f2 P 8802ca5e37c14932aa72e51b6ca84122: Deleting consensus metadata
I20260812 18:59:25.205981 6548 catalog_manager.cc:4841] TS 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271): tablet 5c4986c93adb4f97bb75a450364f97f2 (table test-table [id=5a8eecd62401419cbf2b3762c087f314]) successfully deleted
I20260812 18:59:25.206073 6547 catalog_manager.cc:4841] TS 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013): tablet 5c4986c93adb4f97bb75a450364f97f2 (table test-table [id=5a8eecd62401419cbf2b3762c087f314]) successfully deleted
I20260812 18:59:25.206352 6550 catalog_manager.cc:4841] TS 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195:42069): tablet 5c4986c93adb4f97bb75a450364f97f2 (table test-table [id=5a8eecd62401419cbf2b3762c087f314]) successfully deleted
I20260812 18:59:25.207108 6569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:59:25.207520 6569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:59:25.212721 7070 tablet_service.cc:1467] Processing CreateTablet for tablet f835c594c3a3443dadaaf8f7c7855221 (DEFAULT_TABLE table=test-table [id=016dceb63a7a4832bc2430fd1a9555d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:25.213011 7070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f835c594c3a3443dadaaf8f7c7855221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:25.212970 6732 tablet_service.cc:1467] Processing CreateTablet for tablet f835c594c3a3443dadaaf8f7c7855221 (DEFAULT_TABLE table=test-table [id=016dceb63a7a4832bc2430fd1a9555d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:25.212970 6904 tablet_service.cc:1467] Processing CreateTablet for tablet f835c594c3a3443dadaaf8f7c7855221 (DEFAULT_TABLE table=test-table [id=016dceb63a7a4832bc2430fd1a9555d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:25.213197 6732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f835c594c3a3443dadaaf8f7c7855221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:25.213197 6904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f835c594c3a3443dadaaf8f7c7855221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:25.215235 7189 tablet_bootstrap.cc:492] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6: Bootstrap starting.
I20260812 18:59:25.215245 7188 tablet_bootstrap.cc:492] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122: Bootstrap starting.
I20260812 18:59:25.215960 7188 tablet_bootstrap.cc:654] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:25.215960 7189 tablet_bootstrap.cc:654] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:25.216473 7191 tablet_bootstrap.cc:492] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479: Bootstrap starting.
I20260812 18:59:25.217190 7191 tablet_bootstrap.cc:654] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:25.218300 7191 tablet_bootstrap.cc:492] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479: No bootstrap required, opened a new log
I20260812 18:59:25.218381 7191 ts_tablet_manager.cc:1397] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 18:59:25.218515 7189 tablet_bootstrap.cc:492] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6: No bootstrap required, opened a new log
I20260812 18:59:25.218583 7189 ts_tablet_manager.cc:1397] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 18:59:25.218758 7191 raft_consensus.cc:348] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.218854 7191 raft_consensus.cc:374] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:25.218886 7191 raft_consensus.cc:729] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412285c987324f11bebdb6a60cb46479, State: Initialized, Role: FOLLOWER
I20260812 18:59:25.218926 7189 raft_consensus.cc:348] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.219017 7189 raft_consensus.cc:374] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:25.219048 7189 raft_consensus.cc:729] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f47983fbe1b46ed9036354e0ed776b6, State: Initialized, Role: FOLLOWER
I20260812 18:59:25.219015 7191 consensus_queue.cc:260] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.219157 7189 consensus_queue.cc:260] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.219283 7191 ts_tablet_manager.cc:1428] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 18:59:25.219398 7189 ts_tablet_manager.cc:1428] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:25.219830 7188 tablet_bootstrap.cc:492] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122: No bootstrap required, opened a new log
I20260812 18:59:25.219915 7188 ts_tablet_manager.cc:1397] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 18:59:25.220297 7188 raft_consensus.cc:348] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.220417 7188 raft_consensus.cc:374] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:25.220453 7188 raft_consensus.cc:729] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8802ca5e37c14932aa72e51b6ca84122, State: Initialized, Role: FOLLOWER
I20260812 18:59:25.220594 7188 consensus_queue.cc:260] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.220870 7188 ts_tablet_manager.cc:1428] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 18:59:25.400451 7260 raft_consensus.cc:482] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:59:25.400665 7260 raft_consensus.cc:504] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.401083 7260 leader_election.cc:288] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013), 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271)
I20260812 18:59:25.401582 6927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f835c594c3a3443dadaaf8f7c7855221" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412285c987324f11bebdb6a60cb46479" is_pre_election: true
I20260812 18:59:25.401582 6741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f835c594c3a3443dadaaf8f7c7855221" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f47983fbe1b46ed9036354e0ed776b6" is_pre_election: true
I20260812 18:59:25.401734 6927 raft_consensus.cc:2454] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 0.
I20260812 18:59:25.401736 6741 raft_consensus.cc:2454] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 0.
I20260812 18:59:25.402092 7010 leader_election.cc:302] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 412285c987324f11bebdb6a60cb46479, 6f47983fbe1b46ed9036354e0ed776b6, 8802ca5e37c14932aa72e51b6ca84122; no voters:
I20260812 18:59:25.402297 7260 raft_consensus.cc:2781] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:59:25.402360 7260 raft_consensus.cc:482] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:59:25.402388 7260 raft_consensus.cc:3037] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:25.403402 7260 raft_consensus.cc:504] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.403741 7260 leader_election.cc:288] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 election: Requested vote from peers 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013), 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271)
I20260812 18:59:25.404099 6927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f835c594c3a3443dadaaf8f7c7855221" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412285c987324f11bebdb6a60cb46479"
I20260812 18:59:25.404099 6741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f835c594c3a3443dadaaf8f7c7855221" candidate_uuid: "8802ca5e37c14932aa72e51b6ca84122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f47983fbe1b46ed9036354e0ed776b6"
I20260812 18:59:25.404225 6741 raft_consensus.cc:3037] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:25.404225 6927 raft_consensus.cc:3037] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:25.405359 6927 raft_consensus.cc:2454] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 1.
I20260812 18:59:25.405359 6741 raft_consensus.cc:2454] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8802ca5e37c14932aa72e51b6ca84122 in term 1.
I20260812 18:59:25.405637 7009 leader_election.cc:302] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f47983fbe1b46ed9036354e0ed776b6, 8802ca5e37c14932aa72e51b6ca84122; no voters:
I20260812 18:59:25.405854 7260 raft_consensus.cc:2781] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:25.405956 7260 raft_consensus.cc:686] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 LEADER]: Becoming Leader. State: Replica: 8802ca5e37c14932aa72e51b6ca84122, State: Running, Role: LEADER
I20260812 18:59:25.406098 7260 consensus_queue.cc:237] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } }
I20260812 18:59:25.407804 6567 catalog_manager.cc:5495] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195). New cstate: current_term: 1 leader_uuid: "8802ca5e37c14932aa72e51b6ca84122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8802ca5e37c14932aa72e51b6ca84122" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260812 18:59:25.469758 7186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:25.466483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:46829 (local address 127.29.112.195:42069)
0812 18:59:25.466597 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:25.466601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:25.466776 (+ 175us) server_negotiation.cc:407] Connection header received
0812 18:59:25.466898 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:25.466901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:25.466955 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:25.467018 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:59:25.467214 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:25.467701 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:25.468648 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:25.468860 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:25.468943 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:25.469323 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:59:25.469424 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:59:25.469477 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:25.469606 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 18:59:25.469646 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 18:59:25.469775 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:25.466450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:42069 (local address 127.29.112.195:46829)
0812 18:59:25.466680 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:25.466693 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:25.466787 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:25.467041 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:25.467043 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:25.467049 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 18:59:25.467140 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:25.467144 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:25.467816 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:25.467819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:25.468469 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:25.468479 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:25.469015 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:25.469018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:25.469134 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:25.469191 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 18:59:25.469506 (+ 315us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 18:59:25.469512 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 18:59:25.469579 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 18:59:25.469659 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260812 18:59:25.475035 6741 raft_consensus.cc:1261] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Refusing update from remote peer 8802ca5e37c14932aa72e51b6ca84122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:25.475073 6927 raft_consensus.cc:1261] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Refusing update from remote peer 8802ca5e37c14932aa72e51b6ca84122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:25.475404 7260 consensus_queue.cc:1035] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f47983fbe1b46ed9036354e0ed776b6" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:25.475580 7195 consensus_queue.cc:1035] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "412285c987324f11bebdb6a60cb46479" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 43271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:25.507166 6569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:25.507311 6569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:25.509266 6569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd3a7c5697d74e42bfa309b3d254e1c2: Sending DeleteTablet for 3 replicas of tablet f835c594c3a3443dadaaf8f7c7855221
I20260812 18:59:25.509878 6732 tablet_service.cc:1514] Processing DeleteTablet for tablet f835c594c3a3443dadaaf8f7c7855221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:25 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:55142
I20260812 18:59:25.509905 7070 tablet_service.cc:1514] Processing DeleteTablet for tablet f835c594c3a3443dadaaf8f7c7855221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:25 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:32984
I20260812 18:59:25.509953 6904 tablet_service.cc:1514] Processing DeleteTablet for tablet f835c594c3a3443dadaaf8f7c7855221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:25 UTC) from {username='oryx', principal='oryx/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:34306
I20260812 18:59:25.510054 7273 tablet_replica.cc:330] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122: stopping tablet replica
I20260812 18:59:25.510058 7272 tablet_replica.cc:330] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6: stopping tablet replica
I20260812 18:59:25.510164 7273 raft_consensus.cc:2229] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:59:25.510165 7272 raft_consensus.cc:2229] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:25.510257 7272 raft_consensus.cc:2258] T f835c594c3a3443dadaaf8f7c7855221 P 6f47983fbe1b46ed9036354e0ed776b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:25.510327 7273 raft_consensus.cc:2258] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:25.510497 7271 tablet_replica.cc:330] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479: stopping tablet replica
I20260812 18:59:25.510648 7271 raft_consensus.cc:2229] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:25.510655 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 6658
I20260812 18:59:25.510754 7271 raft_consensus.cc:2258] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:25.515731 7271 ts_tablet_manager.cc:1905] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:25.515743 7273 ts_tablet_manager.cc:1905] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:25.518218 7273 ts_tablet_manager.cc:1918] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:59:25.518321 7273 log.cc:1196] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/f835c594c3a3443dadaaf8f7c7855221
I20260812 18:59:25.518715 7273 ts_tablet_manager.cc:1939] T f835c594c3a3443dadaaf8f7c7855221 P 8802ca5e37c14932aa72e51b6ca84122: Deleting consensus metadata
I20260812 18:59:25.519033 7271 ts_tablet_manager.cc:1918] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 18:59:25.519124 7271 log.cc:1196] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/f835c594c3a3443dadaaf8f7c7855221
I20260812 18:59:25.519438 7271 ts_tablet_manager.cc:1939] T f835c594c3a3443dadaaf8f7c7855221 P 412285c987324f11bebdb6a60cb46479: Deleting consensus metadata
I20260812 18:59:25.519537 6550 catalog_manager.cc:4841] TS 8802ca5e37c14932aa72e51b6ca84122 (127.29.112.195:42069): tablet f835c594c3a3443dadaaf8f7c7855221 (table test-table [id=016dceb63a7a4832bc2430fd1a9555d5]) successfully deleted
I20260812 18:59:25.520118 6548 catalog_manager.cc:4841] TS 412285c987324f11bebdb6a60cb46479 (127.29.112.194:43271): tablet f835c594c3a3443dadaaf8f7c7855221 (table test-table [id=016dceb63a7a4832bc2430fd1a9555d5]) successfully deleted
W20260812 18:59:25.524446 6547 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:45013 (error 108)
I20260812 18:59:25.524493 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 6825
I20260812 18:59:25.528246 7315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:25.527247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45013 (local address 127.0.0.1:58068)
0812 18:59:25.528036 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:25.528123 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":67,"threads_started":1}
W20260812 18:59:25.528388 6547 catalog_manager.cc:4584] TS 6f47983fbe1b46ed9036354e0ed776b6 (127.29.112.193:45013): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f835c594c3a3443dadaaf8f7c7855221: Network error: Client connection negotiation failed: client connection to 127.29.112.193:45013: connect: Connection refused (error 111)
I20260812 18:59:25.535302 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 6991
I20260812 18:59:25.544621 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 6530
2026-08-12T18:59:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3661 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:59:25 dist-test-slave-6nmv krb5kdc[7318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:59:25 dist-test-slave-6nmv krb5kdc[7318](info): set up 2 sockets
Aug 12 18:59:25 dist-test-slave-6nmv krb5kdc[7318](info): commencing operation
krb5kdc: starting...
W20260812 18:59:26.642745 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:26 dist-test-slave-6nmv krb5kdc[7318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:59:26.764885 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:34289
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:45429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:34289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:26.870820 7331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:26.871086 7331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:26.871145 7331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:26.874976 7331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:26.875021 7331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:26.875038 7331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:26.875072 7331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:26.875087 7331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:26.879877 7331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:45429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:34289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:34289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:26.880685 7331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:26.881512 7331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:26.889449 7336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:26.889420 7341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:26.889463 7337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:26.889854 7331 server_base.cc:1034] running on GCE node
I20260812 18:59:26.890398 7331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:26.891350 7331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:26.892539 7331 hybrid_clock.cc:648] HybridClock initialized: now 1786561166892523 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T18:59:26Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1352 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:59:26 dist-test-slave-6nmv krb5kdc[7347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:59:26 dist-test-slave-6nmv krb5kdc[7347](info): set up 2 sockets
Aug 12 18:59:26 dist-test-slave-6nmv krb5kdc[7347](info): commencing operation
krb5kdc: starting...
W20260812 18:59:28.002458 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:28 dist-test-slave-6nmv krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:59:28.129309 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38329
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:38329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:28.238485 7361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:28.238778 7361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:28.238857 7361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:28.242808 7361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:28.242901 7361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:28.242944 7361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:28.243005 7361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:28.243049 7361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:28.248273 7361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:38329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:28.249291 7361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:28.250353 7361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:28.258728 7366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:28.258766 7368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:28.258934 7361 server_base.cc:1034] running on GCE node
W20260812 18:59:28.258755 7371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:28.259590 7361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:28.260644 7361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:28.261873 7361 hybrid_clock.cc:648] HybridClock initialized: now 1786561168261856 us; error 37 us; skew 500 ppm
Aug 12 18:59:28 dist-test-slave-6nmv krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561168, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:28.264490 7361 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:59:28.265280 7361 webserver.cc:460] Webserver started at http://127.29.112.254:41149/ using document root <none> and password file <none>
I20260812 18:59:28.265797 7361 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:28.265985 7361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:28.266373 7361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:28.268419 7361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "e6567845e1c5459f93ab0c4f48987951"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "391995c1c09db7e9bf4aa3f0ead44ea4"
server_key_iv: "19e60ecef99383f507fa37d306d25eeb"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.268959 7361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "e6567845e1c5459f93ab0c4f48987951"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "391995c1c09db7e9bf4aa3f0ead44ea4"
server_key_iv: "19e60ecef99383f507fa37d306d25eeb"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.272524 7361 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 18:59:28.274890 7378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:28.276027 7361 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:28.276158 7361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "e6567845e1c5459f93ab0c4f48987951"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "391995c1c09db7e9bf4aa3f0ead44ea4"
server_key_iv: "19e60ecef99383f507fa37d306d25eeb"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.276257 7361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:28.285650 7361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:28.288321 7361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:28.288506 7361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:28.295416 7361 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:38329
I20260812 18:59:28.295451 7440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:38329 every 8 connection(s)
I20260812 18:59:28.296285 7361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:28.297183 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 7361
I20260812 18:59:28.297317 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:28.297645 30147 external_mini_cluster.cc:1428] Setting key 1333bfebeab79dc3956089dac0fe648e
I20260812 18:59:28.299399 7441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:28.305620 7441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951: Bootstrap starting.
Aug 12 18:59:28 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:28.308204 7441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:28.309092 7441 log.cc:824] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:28.311309 7441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951: No bootstrap required, opened a new log
I20260812 18:59:28.312071 7445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:28.299005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.29.112.254:38329)
0812 18:59:28.299388 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:28.299403 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:28.299437 (+ 34us) server_negotiation.cc:407] Connection header received
0812 18:59:28.300082 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:28.300097 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:28.300383 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:28.300642 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:28.301640 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:28.302582 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:28.303698 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:28.304108 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:28.304222 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:28.306931 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:28.306938 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:28.306953 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:28.306983 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:28.309035 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:28.309493 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:28.309499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:28.309504 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:28.309637 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:28.310016 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:28.310022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:28.310026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:28.310495 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:28.310695 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:28.310908 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 18:59:28.311183 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":127,"threads_started":1}
I20260812 18:59:28.315522 7441 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } }
I20260812 18:59:28.315739 7441 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:28.315776 7441 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6567845e1c5459f93ab0c4f48987951, State: Initialized, Role: FOLLOWER
I20260812 18:59:28.316109 7441 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } }
I20260812 18:59:28.316220 7441 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:28.316268 7441 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:28.316339 7441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:28.317160 7441 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } }
I20260812 18:59:28.317445 7441 leader_election.cc:302] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6567845e1c5459f93ab0c4f48987951; no voters:
I20260812 18:59:28.317715 7441 leader_election.cc:288] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:28.317819 7448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:28.318105 7448 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 1 LEADER]: Becoming Leader. State: Replica: e6567845e1c5459f93ab0c4f48987951, State: Running, Role: LEADER
I20260812 18:59:28.318506 7448 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } }
I20260812 18:59:28.319007 7441 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:28.320006 7449 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6567845e1c5459f93ab0c4f48987951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } } }
I20260812 18:59:28.320130 7449 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:28.320519 7457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:28.320487 7450 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6567845e1c5459f93ab0c4f48987951. Latest consensus state: current_term: 1 leader_uuid: "e6567845e1c5459f93ab0c4f48987951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } } }
I20260812 18:59:28.320575 7450 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:28.323585 7457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:28.329382 7457 catalog_manager.cc:1347] Generated new cluster ID: 6e574d346a2f46f59e44f760552442c1
I20260812 18:59:28.329454 7457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:28.347159 7457 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:28.348050 7457 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:28.352470 7457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951: Generated new TSK 0
I20260812 18:59:28.353114 7457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 18:59:28.383078 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38329
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:28.492940 7470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:28.493207 7470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:28.493290 7470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:28.497265 7470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:28.497345 7470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:28.497432 7470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:28.502544 7470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:28.503505 7470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:28.504441 7470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:28.513734 7477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:28.513794 7481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:28.513794 7479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:28.513950 7470 server_base.cc:1034] running on GCE node
I20260812 18:59:28.514478 7470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:28.515030 7470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:28.516239 7470 hybrid_clock.cc:648] HybridClock initialized: now 1786561168516222 us; error 33 us; skew 500 ppm
Aug 12 18:59:28 dist-test-slave-6nmv krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561168, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:28.519063 7470 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 18:59:28.519976 7470 webserver.cc:460] Webserver started at http://127.29.112.193:45065/ using document root <none> and password file <none>
I20260812 18:59:28.520512 7470 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:28.520570 7470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:28.520771 7470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:28.522759 7470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "f6b43d9666784bbb94f2ea0026b7fabd"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "0696442f705c790601f39cf3d4cdd835"
server_key_iv: "a1f3a6aa05331fad2b68ceb80943c44b"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.523301 7470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "f6b43d9666784bbb94f2ea0026b7fabd"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "0696442f705c790601f39cf3d4cdd835"
server_key_iv: "a1f3a6aa05331fad2b68ceb80943c44b"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.526862 7470 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260812 18:59:28.529150 7488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:28.530396 7470 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 18:59:28.530486 7470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "f6b43d9666784bbb94f2ea0026b7fabd"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "0696442f705c790601f39cf3d4cdd835"
server_key_iv: "a1f3a6aa05331fad2b68ceb80943c44b"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.530563 7470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:28.549558 7470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:28.552227 7470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:28.552366 7470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:28.552845 7470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:28.553555 7470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:28.553607 7470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:28.553644 7470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:28.553664 7470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:28.562304 7470 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:34949
I20260812 18:59:28.562348 7621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:34949 every 8 connection(s)
I20260812 18:59:28.563120 7470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:28.570621 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 7470
I20260812 18:59:28.570746 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:28.571053 30147 external_mini_cluster.cc:1428] Setting key 2cbc6e055a76532c2bd9b6d9fee7f21f
Aug 12 18:59:28 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561168, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:28.576768 7445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:28.564778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:58675 (local address 127.29.112.254:38329)
0812 18:59:28.564945 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:28.564950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:28.565766 (+ 816us) server_negotiation.cc:407] Connection header received
0812 18:59:28.567044 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:28.567050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:28.567098 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:28.567191 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:28.568274 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:28.568784 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:28.570042 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:28.570294 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:28.570376 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:28.573066 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:28.573072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:28.573077 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:28.573107 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:28.574702 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:28.575283 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:28.575285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:28.575287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:28.575354 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:28.575723 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:28.575725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:28.575726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:28.575873 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:28.575941 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:28.576548 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 18:59:28.576657 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 18:59:28.577978 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:28.565156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:58675)
0812 18:59:28.565624 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:28.565662 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:28.566848 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:28.567350 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:28.567356 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:28.567690 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:28.568125 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:28.568137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:28.568900 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:28.568903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:28.569937 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:28.569945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:28.570463 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:28.570469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:28.570665 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:28.571256 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:28.571275 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:28.572891 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:28.574827 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:28.574832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:28.574847 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:28.575191 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:28.575447 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:28.575452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:28.575454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:28.575630 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:28.575975 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:28.575983 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:28.576263 (+ 280us) client_negotiation.cc:769] Sending connection context
0812 18:59:28.576490 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 18:59:28.576840 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":133,"threads_started":1}
I20260812 18:59:28.579255 7622 heartbeater.cc:347] Connected to a master server at 127.29.112.254:38329
I20260812 18:59:28.579491 7622 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:28.580003 7622 heartbeater.cc:511] Master 127.29.112.254:38329 requested a full tablet report, sending...
I20260812 18:59:28.581640 7398 ts_manager.cc:194] Registered new tserver with Master: f6b43d9666784bbb94f2ea0026b7fabd (127.29.112.193:34949)
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 18:59:28.582990 7398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:58675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 18:59:28.598685 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38329
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:28.705641 7627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:28.705914 7627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:28.705984 7627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:28.709666 7627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:28.709735 7627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:28.709869 7627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:28.714772 7627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:28.715638 7627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:28.716519 7627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:28.725459 7636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:28.725518 7633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:28.725545 7634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:28.725642 7627 server_base.cc:1034] running on GCE node
I20260812 18:59:28.726207 7627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:28.726763 7627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:28.727916 7627 hybrid_clock.cc:648] HybridClock initialized: now 1786561168727900 us; error 31 us; skew 500 ppm
Aug 12 18:59:28 dist-test-slave-6nmv krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561168, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:28.730634 7627 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 18:59:28.731389 7627 webserver.cc:460] Webserver started at http://127.29.112.194:44963/ using document root <none> and password file <none>
I20260812 18:59:28.731940 7627 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:28.732033 7627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:28.732224 7627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:28.734256 7627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "f75ed257236547f9b1ecb86cf6e23a93"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "1a3a03d7601d3df0c28d8a71d442894d"
server_key_iv: "9617b2d1ed5be24187e615bdcb38fcce"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.734851 7627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "f75ed257236547f9b1ecb86cf6e23a93"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "1a3a03d7601d3df0c28d8a71d442894d"
server_key_iv: "9617b2d1ed5be24187e615bdcb38fcce"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.738335 7627 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 18:59:28.740650 7643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:28.741721 7627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:28.741878 7627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "f75ed257236547f9b1ecb86cf6e23a93"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "1a3a03d7601d3df0c28d8a71d442894d"
server_key_iv: "9617b2d1ed5be24187e615bdcb38fcce"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.741984 7627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:28.761965 7627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:28.764652 7627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:28.764858 7627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:28.765406 7627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:28.766266 7627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:28.766332 7627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:28.766396 7627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:28.766435 7627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:28.775060 7627 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:33395
I20260812 18:59:28.775122 7790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:33395 every 8 connection(s)
I20260812 18:59:28.775889 7627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
Aug 12 18:59:28 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561168, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:28.785248 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 7627
I20260812 18:59:28.785403 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:28.785689 30147 external_mini_cluster.cc:1428] Setting key 301029fd4a3717dae8a7a05bfe68a367
I20260812 18:59:28.788874 7445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:28.777557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:41417 (local address 127.29.112.254:38329)
0812 18:59:28.777700 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:28.777705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:28.778602 (+ 897us) server_negotiation.cc:407] Connection header received
0812 18:59:28.779876 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:28.779879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:28.779923 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:28.779991 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:28.781038 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:28.781502 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:28.782548 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:28.782754 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:28.782883 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:28.785276 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:28.785279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:28.785281 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:28.785302 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:28.786819 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:28.787371 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:28.787376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:28.787378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:28.787461 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:28.787803 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:28.787808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:28.787809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:28.787990 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:28.788097 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:28.788599 (+ 502us) server_negotiation.cc:299] Negotiation successful
0812 18:59:28.788713 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 18:59:28.789701 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:28.778024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:41417)
0812 18:59:28.778450 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:28.778490 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:28.779695 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:28.780194 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:28.780206 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:28.780511 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:28.780919 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:28.780931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:28.781604 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:28.781606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:28.782438 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:28.782447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:28.782918 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:28.782921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:28.783043 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:28.783624 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:28.783656 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:28.785104 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:28.786958 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:28.786963 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:28.786975 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:28.787265 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:28.787563 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:28.787567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:28.787569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:28.787707 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:28.788095 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:28.788101 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:28.788347 (+ 246us) client_negotiation.cc:769] Sending connection context
0812 18:59:28.788547 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 18:59:28.788844 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":105,"threads_started":1}
I20260812 18:59:28.790781 7791 heartbeater.cc:347] Connected to a master server at 127.29.112.254:38329
I20260812 18:59:28.791020 7791 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:28.791580 7791 heartbeater.cc:511] Master 127.29.112.254:38329 requested a full tablet report, sending...
I20260812 18:59:28.792755 7398 ts_manager.cc:194] Registered new tserver with Master: f75ed257236547f9b1ecb86cf6e23a93 (127.29.112.194:33395)
I20260812 18:59:28.793354 7398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:41417
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 18:59:28.814409 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38329
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:28.922475 7796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:28.922675 7796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:28.922750 7796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:28.926326 7796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:28.926370 7796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:28.926468 7796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:28.931548 7796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:28.932524 7796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:28.933511 7796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:28.942720 7805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:28.942783 7802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:28.942888 7803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:28.942920 7796 server_base.cc:1034] running on GCE node
I20260812 18:59:28.943400 7796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:28.943953 7796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:28.945104 7796 hybrid_clock.cc:648] HybridClock initialized: now 1786561168945083 us; error 33 us; skew 500 ppm
Aug 12 18:59:28 dist-test-slave-6nmv krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561168, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:28.947762 7796 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 18:59:28.948629 7796 webserver.cc:460] Webserver started at http://127.29.112.195:40935/ using document root <none> and password file <none>
I20260812 18:59:28.949184 7796 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:28.949275 7796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:28.949506 7796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:28.951382 7796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "35cc4957d6664ace9fedde4c0148f268"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "0ebab5c17434f4b44d05bf10a267b84f"
server_key_iv: "8faf1f2bf5a0f427ac007d60a85272cd"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.951972 7796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "35cc4957d6664ace9fedde4c0148f268"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "0ebab5c17434f4b44d05bf10a267b84f"
server_key_iv: "8faf1f2bf5a0f427ac007d60a85272cd"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.955487 7796 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 18:59:28.957998 7814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:28.959201 7796 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:28.959347 7796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "35cc4957d6664ace9fedde4c0148f268"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "0ebab5c17434f4b44d05bf10a267b84f"
server_key_iv: "8faf1f2bf5a0f427ac007d60a85272cd"
server_key_version: "encryptionkey@0"
I20260812 18:59:28.959457 7796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:28.986965 7796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:28.989748 7796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:28.989991 7796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:28.990615 7796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:28.991428 7796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:28.991503 7796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:28.991575 7796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:28.991631 7796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:29.000140 7796 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:38437
I20260812 18:59:29.000169 7950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:38437 every 8 connection(s)
I20260812 18:59:29.001006 7796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561168, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:29.010974 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 7796
I20260812 18:59:29.011111 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:29.011443 30147 external_mini_cluster.cc:1428] Setting key 24909feb5e1ede9e672f953a884d9265
I20260812 18:59:29.014520 7445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.002612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:40301 (local address 127.29.112.254:38329)
0812 18:59:29.002734 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:29.002738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:29.003420 (+ 682us) server_negotiation.cc:407] Connection header received
0812 18:59:29.004719 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:29.004722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:29.004781 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:29.004880 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:29.006035 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.006576 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.008117 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.008348 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.008455 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.010802 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:29.010809 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:29.010811 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:29.010840 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:29.012483 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:29.013086 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:29.013090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:29.013091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:29.013181 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:29.013500 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:29.013502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:29.013504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:29.013736 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:29.013863 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:29.014314 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 18:59:29.014406 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 18:59:29.015239 7953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.002868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:40301)
0812 18:59:29.003257 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:29.003313 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:29.004503 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:29.005043 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:29.005050 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:29.005360 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:29.005899 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.005915 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.006741 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.006745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.007981 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.007991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.008501 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.008505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.008658 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.009252 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:29.009266 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:29.010642 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:29.012600 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:29.012608 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:29.012623 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:29.012981 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:29.013266 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:29.013268 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:29.013270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:29.013385 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:29.013882 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:29.013888 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:29.014118 (+ 230us) client_negotiation.cc:769] Sending connection context
0812 18:59:29.014267 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 18:59:29.014495 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":104,"threads_started":1}
I20260812 18:59:29.016291 7951 heartbeater.cc:347] Connected to a master server at 127.29.112.254:38329
I20260812 18:59:29.016491 7951 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:29.016894 7951 heartbeater.cc:511] Master 127.29.112.254:38329 requested a full tablet report, sending...
I20260812 18:59:29.017792 7398 ts_manager.cc:194] Registered new tserver with Master: 35cc4957d6664ace9fedde4c0148f268 (127.29.112.195:38437)
I20260812 18:59:29.018375 7398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:40301
I20260812 18:59:29.025127 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab.
Entry for principal oryx/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab.
WARNING: no policy specified for oryx/127.29.112.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab.
Entry for principal oryx/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab.
WARNING: no policy specified for oryx/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab.
Entry for principal oryx/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab.
I20260812 18:59:29.096583 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 7470
I20260812 18:59:29.106314 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 7627
I20260812 18:59:29.113497 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 7796
I20260812 18:59:29.124174 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 7361
I20260812 18:59:29.131999 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38329
--webserver_interface=127.29.112.254
--webserver_port=41149
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:38329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:29.238970 7961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:29.239203 7961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:29.239277 7961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:29.243273 7961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:29.243325 7961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:29.243345 7961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:29.243359 7961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:29.243373 7961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:29.248435 7961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:38329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=41149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:29.249401 7961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:29.250458 7961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:29.258544 7968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:29.258602 7967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:29.258544 7971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:29.258896 7961 server_base.cc:1034] running on GCE node
I20260812 18:59:29.259472 7961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:29.260511 7961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:29.261708 7961 hybrid_clock.cc:648] HybridClock initialized: now 1786561169261699 us; error 31 us; skew 500 ppm
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561169, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:29.264457 7961 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:59:29.265262 7961 webserver.cc:460] Webserver started at http://127.29.112.254:41149/ using document root <none> and password file <none>
I20260812 18:59:29.265791 7961 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:29.265944 7961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:29.269556 7961 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:29.271658 7978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:29.272908 7961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 18:59:29.273042 7961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "e6567845e1c5459f93ab0c4f48987951"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "391995c1c09db7e9bf4aa3f0ead44ea4"
server_key_iv: "19e60ecef99383f507fa37d306d25eeb"
server_key_version: "encryptionkey@0"
I20260812 18:59:29.273533 7961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:29.315944 7961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:29.318848 7961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:29.319051 7961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:29.326038 7961 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:38329
I20260812 18:59:29.326056 8037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:38329 every 8 connection(s)
I20260812 18:59:29.327055 7961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:29.328806 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 7961
I20260812 18:59:29.329200 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:34949
--local_ip_for_outbound_sockets=127.29.112.193
--tserver_master_addrs=127.29.112.254:38329
--webserver_port=45065
--webserver_interface=127.29.112.193
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.112.193
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 18:59:29.332154 8038 sys_catalog.cc:263] Verifying existing consensus state
I20260812 18:59:29.334895 8038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951: Bootstrap starting.
I20260812 18:59:29.360234 8038 log.cc:824] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:29.369522 8038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 18:59:29.370178 8038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951: Bootstrap complete.
I20260812 18:59:29.376837 8038 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } }
I20260812 18:59:29.377507 8038 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6567845e1c5459f93ab0c4f48987951, State: Initialized, Role: FOLLOWER
I20260812 18:59:29.378172 8038 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } }
I20260812 18:59:29.378357 8038 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:29.378450 8038 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:29.378590 8038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 1 FOLLOWER]: Advancing to term 2
I20260812 18:59:29.381541 8038 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } }
I20260812 18:59:29.382050 8038 leader_election.cc:302] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6567845e1c5459f93ab0c4f48987951; no voters:
I20260812 18:59:29.382475 8038 leader_election.cc:288] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 18:59:29.382607 8042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 18:59:29.382884 8042 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [term 2 LEADER]: Becoming Leader. State: Replica: e6567845e1c5459f93ab0c4f48987951, State: Running, Role: LEADER
I20260812 18:59:29.383181 8042 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } }
I20260812 18:59:29.383893 8038 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:29.384084 8044 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6567845e1c5459f93ab0c4f48987951. Latest consensus state: current_term: 2 leader_uuid: "e6567845e1c5459f93ab0c4f48987951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } } }
I20260812 18:59:29.384213 8044 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:29.384464 8043 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e6567845e1c5459f93ab0c4f48987951" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6567845e1c5459f93ab0c4f48987951" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38329 } } }
I20260812 18:59:29.384543 8043 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:29.384869 8046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:29.388176 8046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:29.389007 8046 catalog_manager.cc:1259] Loaded cluster ID: 6e574d346a2f46f59e44f760552442c1
I20260812 18:59:29.389070 8046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:29.391407 8046 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:29.392078 8046 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e6567845e1c5459f93ab0c4f48987951: Loaded TSK: 0
I20260812 18:59:29.393250 8046 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 18:59:29.463195 8040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:29.463449 8040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:29.463523 8040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:29.467519 8040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:29.467605 8040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:29.467695 8040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:29.472944 8040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.193.keytab
--principal=oryx/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:34949
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=45065
--tserver_master_addrs=127.29.112.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:29.473994 8040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:29.474998 8040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:29.484277 8068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:29.484282 8067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:29.484342 8070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:29.484794 8040 server_base.cc:1034] running on GCE node
I20260812 18:59:29.485303 8040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:29.486093 8040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:29.487301 8040 hybrid_clock.cc:648] HybridClock initialized: now 1786561169487278 us; error 31 us; skew 500 ppm
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:29.490653 8040 init.cc:377] Logged in from keytab as oryx/127.29.112.193@KRBTEST.COM (short username oryx)
I20260812 18:59:29.491505 8040 webserver.cc:460] Webserver started at http://127.29.112.193:45065/ using document root <none> and password file <none>
I20260812 18:59:29.492106 8040 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:29.492187 8040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:29.496026 8040 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 18:59:29.498222 8078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:29.499451 8040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 18:59:29.499585 8040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "f6b43d9666784bbb94f2ea0026b7fabd"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "0696442f705c790601f39cf3d4cdd835"
server_key_iv: "a1f3a6aa05331fad2b68ceb80943c44b"
server_key_version: "encryptionkey@0"
I20260812 18:59:29.500106 8040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:29.522521 8040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:29.525837 8040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:29.526044 8040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:29.526635 8040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:29.527475 8040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:29.527547 8040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:29.527627 8040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:29.527678 8040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:29.536468 8040 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:34949
I20260812 18:59:29.536540 8208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:34949 every 8 connection(s)
I20260812 18:59:29.537494 8040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:29.547392 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 8040
I20260812 18:59:29.547947 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:33395
--local_ip_for_outbound_sockets=127.29.112.194
--tserver_master_addrs=127.29.112.254:38329
--webserver_port=44963
--webserver_interface=127.29.112.194
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.112.194
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:29.550822 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.539712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:51297)
0812 18:59:29.540228 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:29.540269 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:29.541580 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:29.542684 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:29.542691 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:29.543104 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:29.543530 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.543543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.544753 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.544757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.545574 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.545584 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.546376 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.546380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.546580 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.547163 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:29.547202 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:29.549921 (+ 2719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":151,"threads_started":1}
W20260812 18:59:29.550813 8214 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:29.539586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:51297 (local address 127.29.112.254:38329)
0812 18:59:29.540072 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:29.540094 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:29.540375 (+ 281us) server_negotiation.cc:407] Connection header received
0812 18:59:29.542002 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:29.542015 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:29.542300 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:29.542625 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:29.543707 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.544620 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.545700 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.546214 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.546384 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.550122 (+ 3738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:51297: BlockingRecv error: recv got EOF from 127.29.112.193:51297 (error 108)
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":121,"threads_started":1}
W20260812 18:59:29.551440 8210 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:38329 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.112.254:38329: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:29.556916 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.551952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:41423)
0812 18:59:29.552081 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:29.552099 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:29.552179 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:29.552426 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:29.552429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:29.552585 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:29.552675 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.552680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.553412 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.553417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.554000 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.554008 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.554441 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.554445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.554540 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.555021 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:29.555033 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:29.556729 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 18:59:29.556939 8214 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:29.552041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:41423 (local address 127.29.112.254:38329)
0812 18:59:29.552160 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:29.552164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:29.552177 (+ 13us) server_negotiation.cc:407] Connection header received
0812 18:59:29.552274 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:29.552276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:29.552323 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:29.552411 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:29.552788 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.553304 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.554121 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.554333 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.554441 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.556841 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:41423: BlockingRecv error: recv got EOF from 127.29.112.193:41423 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:29.562747 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.557499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:38569)
0812 18:59:29.557675 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:29.557690 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:29.557782 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:29.558067 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:29.558070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:29.558246 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:29.558361 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.558371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.559055 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.559061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.559951 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.559961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.560358 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.560362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.560441 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.560837 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:29.560853 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:29.562636 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98}
W20260812 18:59:29.562911 8210 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 18:59:29.563074 8214 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:29.557571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:38569 (local address 127.29.112.254:38329)
0812 18:59:29.557680 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:29.557683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:29.557751 (+ 68us) server_negotiation.cc:407] Connection header received
0812 18:59:29.557898 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:29.557903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:29.557948 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:29.558008 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:29.558503 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.558941 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.560071 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.560261 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.560331 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.562783 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:38569: BlockingRecv error: recv got EOF from 127.29.112.193:38569 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 18:59:29.656423 8216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:29.656653 8216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:29.656703 8216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:29.660701 8216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:29.660753 8216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:29.660837 8216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:29.665683 8216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.194.keytab
--principal=oryx/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:33395
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=44963
--tserver_master_addrs=127.29.112.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:29.666767 8216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:29.667747 8216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:29.677099 8223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:29.677098 8224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:29.677225 8226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:29.677577 8216 server_base.cc:1034] running on GCE node
I20260812 18:59:29.678040 8216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:29.678658 8216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:29.679828 8216 hybrid_clock.cc:648] HybridClock initialized: now 1786561169679813 us; error 41 us; skew 500 ppm
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:29.682629 8216 init.cc:377] Logged in from keytab as oryx/127.29.112.194@KRBTEST.COM (short username oryx)
I20260812 18:59:29.683457 8216 webserver.cc:460] Webserver started at http://127.29.112.194:44963/ using document root <none> and password file <none>
I20260812 18:59:29.683985 8216 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:29.684095 8216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:29.687821 8216 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 18:59:29.689893 8236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:29.691007 8216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:29.691130 8216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "f75ed257236547f9b1ecb86cf6e23a93"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "1a3a03d7601d3df0c28d8a71d442894d"
server_key_iv: "9617b2d1ed5be24187e615bdcb38fcce"
server_key_version: "encryptionkey@0"
I20260812 18:59:29.691639 8216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:29.715617 8216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:29.718982 8216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:29.719187 8216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:29.719792 8216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:29.720571 8216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:29.720647 8216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:29.720745 8216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:29.720793 8216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:29.729560 8216 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:33395
I20260812 18:59:29.729584 8368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:33395 every 8 connection(s)
I20260812 18:59:29.730479 8216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:29.735427 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 8216
I20260812 18:59:29.735908 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:38437
--local_ip_for_outbound_sockets=127.29.112.195
--tserver_master_addrs=127.29.112.254:38329
--webserver_port=40935
--webserver_interface=127.29.112.195
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.112.195
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 18:59:29.742388 8214 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:29.732173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:55585 (local address 127.29.112.254:38329)
0812 18:59:29.732305 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:29.732309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:29.733158 (+ 849us) server_negotiation.cc:407] Connection header received
0812 18:59:29.734452 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:29.734456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:29.734516 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:29.734628 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:29.736213 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.736727 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.738303 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.738710 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.738822 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.742226 (+ 3404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:55585: BlockingRecv error: recv got EOF from 127.29.112.194:55585 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 18:59:29.742806 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.732546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:55585)
0812 18:59:29.733017 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:29.733056 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:29.734228 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:29.734792 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:29.734798 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:29.735304 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:29.736005 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.736030 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.736877 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.736883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.738182 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.738192 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.738855 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.738858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.738989 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.739575 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:29.739590 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:29.742035 (+ 2445us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":129,"threads_started":1}
W20260812 18:59:29.743345 8370 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:38329 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.112.254:38329: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:29.748363 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.743780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:50365)
0812 18:59:29.743864 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:29.743878 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:29.743945 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:29.744200 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:29.744202 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:29.744341 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:29.744417 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.744422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.745082 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.745086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.745574 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.745581 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.746051 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.746054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.746142 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.746592 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:29.746601 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:29.748231 (+ 1630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 18:59:29.748481 8214 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:29.743877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:50365 (local address 127.29.112.254:38329)
0812 18:59:29.744014 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:29.744018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:29.744035 (+ 17us) server_negotiation.cc:407] Connection header received
0812 18:59:29.744065 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:29.744067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:29.744097 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:29.744185 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:29.744529 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.744954 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.745688 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.745947 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.746049 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.748346 (+ 2297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:50365: BlockingRecv error: recv got EOF from 127.29.112.194:50365 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:29.753727 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.749027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:50747)
0812 18:59:29.749249 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:29.749263 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:29.749373 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:29.749593 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:29.749596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:29.749737 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:29.749794 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.749799 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.750479 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.750483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.750949 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.750956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.751351 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.751354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.751422 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.751901 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:29.751912 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:29.753611 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163}
W20260812 18:59:29.753947 8370 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 18:59:29.754092 8214 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:29.749138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:50747 (local address 127.29.112.254:38329)
0812 18:59:29.749233 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:29.749237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:29.749340 (+ 103us) server_negotiation.cc:407] Connection header received
0812 18:59:29.749468 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:29.749470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:29.749503 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:29.749579 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:29.749921 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.750345 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.751061 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.751250 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.751350 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.753710 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:50747: BlockingRecv error: recv got EOF from 127.29.112.194:50747 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 18:59:29.847838 8374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:29.848147 8374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:29.848223 8374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:29.852370 8374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:29.852447 8374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:29.852532 8374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:29.857609 8374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/krb5kdc/oryx_127.29.112.195.keytab
--principal=oryx/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:38437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=40935
--tserver_master_addrs=127.29.112.254:38329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:29.858659 8374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:29.859622 8374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:29.869067 8384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:29.869067 8382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:29.869089 8381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:29.869426 8374 server_base.cc:1034] running on GCE node
I20260812 18:59:29.869846 8374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:29.870405 8374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:29.871562 8374 hybrid_clock.cc:648] HybridClock initialized: now 1786561169871544 us; error 28 us; skew 500 ppm
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:29.874290 8374 init.cc:377] Logged in from keytab as oryx/127.29.112.195@KRBTEST.COM (short username oryx)
I20260812 18:59:29.875001 8374 webserver.cc:460] Webserver started at http://127.29.112.195:40935/ using document root <none> and password file <none>
I20260812 18:59:29.875469 8374 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:29.875537 8374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:29.879155 8374 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:29.880998 8393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:29.882139 8374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:29.882226 8374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "35cc4957d6664ace9fedde4c0148f268"
format_stamp: "Formatted at 2026-08-12 18:59:28 on dist-test-slave-6nmv"
server_key: "0ebab5c17434f4b44d05bf10a267b84f"
server_key_iv: "8faf1f2bf5a0f427ac007d60a85272cd"
server_key_version: "encryptionkey@0"
I20260812 18:59:29.882777 8374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:29.896982 8374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:29.899737 8374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:29.899923 8374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:29.900465 8374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:29.901230 8374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:29.901275 8374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:29.901338 8374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:29.901376 8374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:29.910295 8374 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:38437
I20260812 18:59:29.910362 8522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:38437 every 8 connection(s)
I20260812 18:59:29.911361 8374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:59:29.913100 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 8374
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 18:59:29.923039 8214 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:29.913625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:51935 (local address 127.29.112.254:38329)
0812 18:59:29.913766 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:29.913770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:29.914715 (+ 945us) server_negotiation.cc:407] Connection header received
0812 18:59:29.916206 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:29.916208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:29.916245 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:29.916302 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:29.917537 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.918095 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.919357 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.919571 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.919677 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.922922 (+ 3245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:51935: BlockingRecv error: recv got EOF from 127.29.112.195:51935 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 18:59:29.923759 8528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.914287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42456 (local address 127.29.112.254:38329)
0812 18:59:29.915080 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:29.915086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:29.915105 (+ 19us) server_negotiation.cc:407] Connection header received
0812 18:59:29.915149 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:29.915155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:29.915211 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:29.915325 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:29.916043 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.916767 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.917619 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.917936 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.918061 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.919469 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:29.919477 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:29.919495 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:29.919523 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:29.921591 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:29.922097 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:29.922102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:29.922108 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:29.922235 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:29.922524 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:29.922527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:29.922529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:29.922881 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:29.923107 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:29.923419 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 18:59:29.923652 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":95,"threads_started":1}
I20260812 18:59:29.923875 8526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.914081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:51935)
0812 18:59:29.914569 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:29.914617 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:29.916011 (+ 1394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:29.916477 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:29.916485 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:29.916923 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:29.917390 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.917404 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.918223 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.918226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.919230 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.919240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.919717 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.919721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.919892 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.920422 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:29.920437 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:29.922818 (+ 2381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":146,"threads_started":1}
W20260812 18:59:29.924388 8523 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:38329 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.112.254:38329: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:29.929670 8526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.924793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:53671)
0812 18:59:29.924881 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:29.924896 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:29.924967 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:29.925289 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:29.925292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:29.925468 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:29.925587 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.925593 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.926394 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.926398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.926953 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.926960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.927407 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.927411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.927498 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.927962 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:29.927978 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:29.929540 (+ 1562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 18:59:29.929751 8528 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:29.924999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:53671 (local address 127.29.112.254:38329)
0812 18:59:29.925120 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:29.925123 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:29.925133 (+ 10us) server_negotiation.cc:407] Connection header received
0812 18:59:29.925159 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:29.925161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:29.925191 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:29.925274 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:29.925715 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.926275 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.927075 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.927299 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.927407 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.929644 (+ 2237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:53671: BlockingRecv error: recv got EOF from 127.29.112.195:53671 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:29 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:29.934728 8526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:29.930244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:38021)
0812 18:59:29.930329 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:29.930361 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:29.930440 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:29.930639 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:29.930641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:29.930787 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:29.930849 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.930853 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.931528 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.931531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.932031 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:29.932038 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.932473 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.932477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:29.932588 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.933038 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:29.933049 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:29.934605 (+ 1556us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 18:59:29.934782 8528 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:29.930300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:38021 (local address 127.29.112.254:38329)
0812 18:59:29.930448 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:29.930452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:29.930466 (+ 14us) server_negotiation.cc:407] Connection header received
0812 18:59:29.930507 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:29.930510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:29.930545 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:29.930617 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:29.930957 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.931417 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.932150 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:29.932366 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:29.932471 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:29.934692 (+ 2221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:38021: BlockingRecv error: recv got EOF from 127.29.112.195:38021 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260812 18:59:29.934928 8523 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 18:59:30 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:30 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:30.569048 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:30.563487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:42265)
0812 18:59:30.563783 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:30.563796 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:30.563882 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:30.564128 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:30.564130 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:30.564309 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:30.564421 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:30.564426 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.565181 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.565184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:30.565779 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:30.565788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.566289 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.566291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:30.566376 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:30.566897 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:30.566909 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:30.568935 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":84,"threads_started":1}
W20260812 18:59:30.569239 8530 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:30.563613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:42265 (local address 127.29.112.254:38329)
0812 18:59:30.563851 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:30.563854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:30.563868 (+ 14us) server_negotiation.cc:407] Connection header received
0812 18:59:30.563971 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:30.563973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:30.564017 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:30.564092 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:30.564526 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.565054 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.565959 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.566179 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.566255 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:30.569071 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:42265: BlockingRecv error: recv got EOF from 127.29.112.193:42265 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":59,"threads_started":1}
Aug 12 18:59:30 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:30 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:30.760146 8532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:30.754473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:43779)
0812 18:59:30.754777 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:30.754792 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:30.754922 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:30.755197 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:30.755200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:30.755421 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:30.755526 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:30.755531 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.756254 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.756259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:30.756791 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:30.756798 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.757288 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.757292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:30.757398 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:30.757963 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:30.757974 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:30.759957 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":80,"threads_started":1}
W20260812 18:59:30.760216 8530 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:30.754590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:43779 (local address 127.29.112.254:38329)
0812 18:59:30.754754 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:30.754759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:30.754871 (+ 112us) server_negotiation.cc:407] Connection header received
0812 18:59:30.755029 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:30.755033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:30.755085 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:30.755171 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:30.755643 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.756147 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.756912 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.757156 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.757267 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:30.760107 (+ 2840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:43779: BlockingRecv error: recv got EOF from 127.29.112.194:43779 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 18:59:30 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:30 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:30.941103 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:30.935577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:36409)
0812 18:59:30.935877 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:30.935891 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:30.935981 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:30.936253 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:30.936256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:30.936507 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:30.936604 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:30.936611 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.937301 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.937303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:30.937963 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:30.937973 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.938446 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.938449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:30.938515 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:30.939113 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:30.939125 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:30.940997 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":87,"threads_started":1}
W20260812 18:59:30.941327 8530 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:30.935656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:36409 (local address 127.29.112.254:38329)
0812 18:59:30.935823 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:30.935828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:30.935950 (+ 122us) server_negotiation.cc:407] Connection header received
0812 18:59:30.936073 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:30.936076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:30.936125 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:30.936191 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:30.936737 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.937205 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.938107 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:30.938335 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:30.938425 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:30.941121 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:36409: BlockingRecv error: recv got EOF from 127.29.112.195:36409 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 12 18:59:31 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:31 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:31.575361 8534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:31.569851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:35039)
0812 18:59:31.570122 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:31.570135 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:31.570221 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:31.570466 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:31.570468 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:31.570633 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:31.570723 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:31.570728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.571710 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.571713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:31.572311 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:31.572320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.572808 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.572812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:31.572880 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:31.573401 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:31.573420 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:31.575248 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
W20260812 18:59:31.575529 8535 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:31.569883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:35039 (local address 127.29.112.254:38329)
0812 18:59:31.570174 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:31.570178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:31.570191 (+ 13us) server_negotiation.cc:407] Connection header received
0812 18:59:31.570317 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:31.570319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:31.570360 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:31.570431 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:31.570837 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.571537 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.572412 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.572635 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.572740 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:31.575369 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:35039: BlockingRecv error: recv got EOF from 127.29.112.193:35039 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":112,"threads_started":1}
Aug 12 18:59:31 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:31 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:31.766993 8537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:31.760925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:43709)
0812 18:59:31.761186 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:31.761199 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:31.761287 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:31.761566 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:31.761569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:31.761781 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:31.761959 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:31.761966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.762694 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.762699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:31.763277 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:31.763285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.763752 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.763756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:31.763840 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:31.764416 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:31.764427 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:31.766771 (+ 2344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":85,"threads_started":1}
W20260812 18:59:31.767009 8535 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:31.761010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:43709 (local address 127.29.112.254:38329)
0812 18:59:31.761157 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:31.761161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:31.761259 (+ 98us) server_negotiation.cc:407] Connection header received
0812 18:59:31.761356 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:31.761358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:31.761399 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:31.761537 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:31.762072 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.762586 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.763400 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.763634 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.763746 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:31.766916 (+ 3170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:43709: BlockingRecv error: recv got EOF from 127.29.112.194:43709 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 18:59:31 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:31 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:31.952103 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:31.941937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:41063)
0812 18:59:31.942230 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:31.942243 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:31.942326 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:31.942553 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:31.942555 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:31.942717 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:31.942804 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:31.942809 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.943559 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.943562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:31.944144 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:31.944151 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.944606 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.944609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:31.944669 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:31.950084 (+ 5415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:31.950098 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:31.951964 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":127,"threads_started":1}
W20260812 18:59:31.952262 8535 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:31.942157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:41063 (local address 127.29.112.254:38329)
0812 18:59:31.942295 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:31.942299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:31.942313 (+ 14us) server_negotiation.cc:407] Connection header received
0812 18:59:31.942400 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:31.942402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:31.942443 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:31.942516 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:31.942912 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.943445 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.944259 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:31.944503 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:31.944591 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:31.952105 (+ 7514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:41063: BlockingRecv error: recv got EOF from 127.29.112.195:41063 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 18:59:32 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:32 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:32.581918 8539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:32.576073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:58865)
0812 18:59:32.576371 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:32.576389 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:32.576502 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:32.576773 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:32.576775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:32.576960 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:32.577057 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:32.577062 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.577931 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.577937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:32.578518 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:32.578526 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.579001 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.579004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:32.579079 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:32.579763 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:32.579780 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:32.581699 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":102,"threads_started":1}
W20260812 18:59:32.581955 8540 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:32.576194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:58865 (local address 127.29.112.254:38329)
0812 18:59:32.576477 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:32.576481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:32.576494 (+ 13us) server_negotiation.cc:407] Connection header received
0812 18:59:32.576618 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:32.576620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:32.576664 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:32.576752 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:32.577156 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.577769 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.578666 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.578896 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.578997 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:32.581811 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:58865: BlockingRecv error: recv got EOF from 127.29.112.193:58865 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":110,"threads_started":1}
Aug 12 18:59:32 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:32 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:32.773483 8541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:32.767804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:55691)
0812 18:59:32.768093 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:32.768107 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:32.768210 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:32.768444 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:32.768446 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:32.768632 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:32.768729 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:32.768734 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.769465 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.769468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:32.770160 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:32.770171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.770600 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.770603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:32.770683 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:32.771190 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:32.771206 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:32.773366 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":74,"threads_started":1}
W20260812 18:59:32.773615 8540 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:32.767932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:55691 (local address 127.29.112.254:38329)
0812 18:59:32.768118 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:32.768122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:32.768176 (+ 54us) server_negotiation.cc:407] Connection header received
0812 18:59:32.768291 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:32.768293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:32.768337 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:32.768406 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:32.768823 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.769340 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.770272 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.770479 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.770553 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:32.773498 (+ 2945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:55691: BlockingRecv error: recv got EOF from 127.29.112.194:55691 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 12 18:59:32 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:32 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:32.958235 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:32.952847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:33869)
0812 18:59:32.953120 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:32.953133 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:32.953273 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:32.953551 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:32.953555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:32.953795 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:32.953948 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:32.953957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.954657 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.954662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:32.955171 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:32.955178 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.955613 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.955617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:32.955694 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:32.956150 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:32.956160 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:32.958073 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":82,"threads_started":1}
W20260812 18:59:32.958305 8540 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:32.952936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:33869 (local address 127.29.112.254:38329)
0812 18:59:32.953086 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:32.953090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:32.953220 (+ 130us) server_negotiation.cc:407] Connection header received
0812 18:59:32.953376 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:32.953379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:32.953429 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:32.953506 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:32.954050 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.954550 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.955286 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:32.955505 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:32.955609 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:32.958198 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:33869: BlockingRecv error: recv got EOF from 127.29.112.195:33869 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 18:59:33 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:33 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:33.588538 8544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:33.582711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:42279)
0812 18:59:33.582950 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:33.582964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:33.583069 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:33.583309 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:33.583311 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:33.583566 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:33.583663 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:33.583669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.584374 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.584376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:33.585027 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:33.585037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.585495 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.585497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:33.585567 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:33.586225 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:33.586238 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:33.588353 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":90,"threads_started":1}
W20260812 18:59:33.588651 8545 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:33.582801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:42279 (local address 127.29.112.254:38329)
0812 18:59:33.583053 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:33.583056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:33.583070 (+ 14us) server_negotiation.cc:407] Connection header received
0812 18:59:33.583148 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:33.583150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:33.583191 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:33.583281 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:33.583782 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.584287 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.585157 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.585393 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.585485 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:33.588504 (+ 3019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:42279: BlockingRecv error: recv got EOF from 127.29.112.193:42279 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":93,"threads_started":1}
Aug 12 18:59:33 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:33 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:33.779332 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:33.774302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:48713)
0812 18:59:33.774593 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:33.774613 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:33.774702 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:33.774903 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:33.774905 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:33.775076 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:33.775167 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:33.775171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.775814 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.775816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:33.776327 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:33.776339 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.776767 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.776769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:33.776827 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:33.777346 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:33.777358 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:33.779231 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":103,"threads_started":1}
W20260812 18:59:33.779455 8545 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:33.774379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:48713 (local address 127.29.112.254:38329)
0812 18:59:33.774545 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:33.774548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:33.774675 (+ 127us) server_negotiation.cc:407] Connection header received
0812 18:59:33.774764 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:33.774766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:33.774806 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:33.774869 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:33.775254 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.775722 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.776439 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.776675 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.776752 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:33.779348 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:48713: BlockingRecv error: recv got EOF from 127.29.112.194:48713 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 18:59:33 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:33 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:33.963868 8548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:33.959011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:35135)
0812 18:59:33.959259 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:33.959272 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:33.959361 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:33.959557 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:33.959559 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:33.959726 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:33.959813 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:33.959817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.960472 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.960474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:33.960998 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:33.961004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.961362 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.961364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:33.961418 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:33.961967 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:33.961977 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:33.963708 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":92,"threads_started":1}
W20260812 18:59:33.963914 8545 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:33.959092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:35135 (local address 127.29.112.254:38329)
0812 18:59:33.959223 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:33.959227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:33.959318 (+ 91us) server_negotiation.cc:407] Connection header received
0812 18:59:33.959421 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:33.959423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:33.959462 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:33.959524 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:33.959904 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.960379 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.961087 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:33.961275 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:33.961343 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:33.963817 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:35135: BlockingRecv error: recv got EOF from 127.29.112.195:35135 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 18:59:34 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:34 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:34.594583 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:34.589443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:44031)
0812 18:59:34.589716 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:34.589729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:34.589829 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:34.590060 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:34.590062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:34.590244 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:34.590337 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:34.590341 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.591041 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.591043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:34.591590 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:34.591596 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.592031 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.592034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:34.592095 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:34.592570 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:34.592581 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:34.594475 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
W20260812 18:59:34.594820 8550 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:34.589523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:44031 (local address 127.29.112.254:38329)
0812 18:59:34.589790 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:34.589794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:34.589848 (+ 54us) server_negotiation.cc:407] Connection header received
0812 18:59:34.589912 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:34.589914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:34.589956 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:34.590024 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:34.590435 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.590952 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.591718 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.591939 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.592012 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:34.594638 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:44031: BlockingRecv error: recv got EOF from 127.29.112.193:44031 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
Aug 12 18:59:34 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:34 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:34.786005 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:34.780227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:39051)
0812 18:59:34.780554 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:34.780571 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:34.780707 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:34.780991 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:34.780994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:34.781209 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:34.781297 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:34.781302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.782105 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.782110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:34.782771 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:34.782796 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.783308 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.783311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:34.783395 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:34.783891 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:34.783901 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:34.785791 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":131,"threads_started":1}
W20260812 18:59:34.786038 8550 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:34.780390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:39051 (local address 127.29.112.254:38329)
0812 18:59:34.780547 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:34.780552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:34.780652 (+ 100us) server_negotiation.cc:407] Connection header received
0812 18:59:34.780812 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:34.780816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:34.780865 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:34.780965 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:34.781443 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.782005 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.782923 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.783202 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.783294 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:34.785946 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:39051: BlockingRecv error: recv got EOF from 127.29.112.194:39051 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 18:59:34 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:34 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:34.970270 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:34.964809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:50825)
0812 18:59:34.965133 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:34.965155 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:34.965284 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:34.965572 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:34.965575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:34.965848 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:34.965969 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:34.965976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.966756 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.966761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:34.967290 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:34.967297 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.967772 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.967776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:34.967872 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:34.968365 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:34.968377 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:34.970115 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":97,"threads_started":1}
W20260812 18:59:34.970321 8550 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:34.964894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:50825 (local address 127.29.112.254:38329)
0812 18:59:34.965098 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:34.965103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:34.965230 (+ 127us) server_negotiation.cc:407] Connection header received
0812 18:59:34.965391 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:34.965396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:34.965462 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:34.965548 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:34.966098 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.966630 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.967425 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:34.967636 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:34.967757 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:34.970227 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:50825: BlockingRecv error: recv got EOF from 127.29.112.195:50825 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 12 18:59:35 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:35 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:35.601532 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:35.595523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:33195)
0812 18:59:35.595847 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:35.595863 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:35.595981 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:35.596237 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:35.596239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:35.596434 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:35.596532 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:35.596538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.597392 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.597395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:35.598201 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:35.598212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.598769 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.598773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:35.598858 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:35.599437 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:35.599449 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:35.601360 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":106,"threads_started":1}
W20260812 18:59:35.601639 8555 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:35.595597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:33195 (local address 127.29.112.254:38329)
0812 18:59:35.595854 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:35.595858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:35.595941 (+ 83us) server_negotiation.cc:407] Connection header received
0812 18:59:35.596072 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:35.596074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:35.596118 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:35.596193 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:35.596637 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.597240 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.598319 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.598647 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.598762 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:35.601491 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:33195: BlockingRecv error: recv got EOF from 127.29.112.193:33195 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":111,"threads_started":1}
Aug 12 18:59:35 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:35 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:35.792029 8556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:35.786799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:41441)
0812 18:59:35.787070 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:35.787083 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:35.787179 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:35.787485 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:35.787490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:35.787725 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:35.787839 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:35.787845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.788583 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.788586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:35.789163 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:35.789172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.789558 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.789561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:35.789621 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:35.790105 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:35.790116 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:35.791916 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":112,"threads_started":1}
W20260812 18:59:35.792110 8555 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:35.786868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:41441 (local address 127.29.112.254:38329)
0812 18:59:35.787024 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:35.787028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:35.787176 (+ 148us) server_negotiation.cc:407] Connection header received
0812 18:59:35.787256 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:35.787258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:35.787300 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:35.787373 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:35.787940 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.788456 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.789269 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.789461 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.789537 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:35.792021 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:41441: BlockingRecv error: recv got EOF from 127.29.112.194:41441 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 18:59:35 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:35 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:35.976418 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:35.971016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:52869)
0812 18:59:35.971284 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:35.971298 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:35.971405 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:35.971632 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:35.971634 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:35.971826 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:35.971922 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:35.971927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.972631 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.972633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:35.973179 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:35.973187 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.973613 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.973616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:35.973682 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:35.974226 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:35.974238 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:35.976273 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":97,"threads_started":1}
W20260812 18:59:35.976586 8555 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:35.971108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:52869 (local address 127.29.112.254:38329)
0812 18:59:35.971258 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:35.971262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:35.971348 (+ 86us) server_negotiation.cc:407] Connection header received
0812 18:59:35.971470 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:35.971472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:35.971517 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:35.971582 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:35.972027 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.972538 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.973285 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:35.973500 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:35.973577 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:35.976431 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:52869: BlockingRecv error: recv got EOF from 127.29.112.195:52869 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 18:59:36 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:36 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:36.609277 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:36.602548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:59539)
0812 18:59:36.602977 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:36.603024 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:36.603193 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:36.603550 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:36.603554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:36.603856 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:36.604016 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:36.604025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.604950 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.604954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:36.605723 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:36.605735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.606338 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.606342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:36.606464 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:36.607166 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:36.607178 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:36.609116 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":107,"threads_started":1}
W20260812 18:59:36.609376 8560 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:36.602628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:59539 (local address 127.29.112.254:38329)
0812 18:59:36.602990 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:36.602995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:36.603114 (+ 119us) server_negotiation.cc:407] Connection header received
0812 18:59:36.603289 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:36.603293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:36.603380 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:36.603492 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:36.604170 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.604784 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.605858 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.606190 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.606289 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:36.609238 (+ 2949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:59539: BlockingRecv error: recv got EOF from 127.29.112.193:59539 (error 108)
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":79,"threads_started":1}
Aug 12 18:59:36 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:36 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:36.798642 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:36.792871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:49351)
0812 18:59:36.793245 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:36.793273 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:36.793416 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:36.793639 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:36.793641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:36.793866 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:36.794006 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:36.794012 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.794735 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.794737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:36.795273 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:36.795280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.795765 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.795768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:36.795831 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:36.796312 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:36.796324 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:36.798508 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":84,"threads_started":1}
W20260812 18:59:36.798745 8560 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:36.793052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:49351 (local address 127.29.112.254:38329)
0812 18:59:36.793187 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:36.793196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:36.793347 (+ 151us) server_negotiation.cc:407] Connection header received
0812 18:59:36.793501 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:36.793511 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:36.793553 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:36.793614 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:36.794106 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.794637 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.795432 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.795670 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.795747 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:36.798624 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:49351: BlockingRecv error: recv got EOF from 127.29.112.194:49351 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 18:59:36 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:36 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:36.986522 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:36.977152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:53495)
0812 18:59:36.977476 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:36.977492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:36.977605 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:36.977867 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:36.977870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:36.978077 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:36.978175 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:36.978181 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.978890 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.978892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:36.979421 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:36.979427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.979858 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.979860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:36.979920 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:36.984029 (+ 4109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:36.984041 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:36.986373 (+ 2332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":107,"threads_started":1}
W20260812 18:59:36.986675 8560 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:36.977159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:53495 (local address 127.29.112.254:38329)
0812 18:59:36.977293 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:36.977297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:36.977578 (+ 281us) server_negotiation.cc:407] Connection header received
0812 18:59:36.977702 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:36.977705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:36.977754 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:36.977832 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:36.978278 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.978800 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.979542 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:36.979761 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:36.979839 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:36.986557 (+ 6718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:53495: BlockingRecv error: recv got EOF from 127.29.112.195:53495 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 18:59:37 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:37 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:37.615657 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:37.610294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:47129)
0812 18:59:37.610597 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:37.610612 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:37.610709 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:37.610932 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:37.610933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:37.611104 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:37.611213 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:37.611219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.611978 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.611980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:37.612525 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:37.612532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.612935 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.612937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:37.612993 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:37.613601 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:37.613613 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:37.615459 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
W20260812 18:59:37.615768 8566 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:37.610425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:47129 (local address 127.29.112.254:38329)
0812 18:59:37.610689 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:37.610693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:37.610706 (+ 13us) server_negotiation.cc:407] Connection header received
0812 18:59:37.610773 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:37.610775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:37.610814 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:37.610886 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:37.611315 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.611867 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.612618 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.612832 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.612932 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:37.615614 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:47129: BlockingRecv error: recv got EOF from 127.29.112.193:47129 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
Aug 12 18:59:37 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:37 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:37.805123 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:37.799492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:58491)
0812 18:59:37.799792 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:37.799805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:37.799935 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:37.800213 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:37.800216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:37.800450 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:37.800546 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:37.800551 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.801446 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.801450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:37.802025 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:37.802034 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.802503 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.802507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:37.802584 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:37.803167 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:37.803178 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:37.805014 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
W20260812 18:59:37.805267 8566 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:37.799549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:58491 (local address 127.29.112.254:38329)
0812 18:59:37.799708 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:37.799712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:37.799879 (+ 167us) server_negotiation.cc:407] Connection header received
0812 18:59:37.800043 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:37.800047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:37.800103 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:37.800184 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:37.800672 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.801327 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.802144 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.802379 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.802487 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:37.805150 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:58491: BlockingRecv error: recv got EOF from 127.29.112.194:58491 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 18:59:37 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:37 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:37.992723 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:37.987345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:51231)
0812 18:59:37.987608 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:37.987631 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:37.987772 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:37.988023 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:37.988025 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:37.988209 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:37.988308 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:37.988314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.989011 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.989014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:37.989543 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:37.989549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.989954 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.989957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:37.990021 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:37.990493 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:37.990506 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:37.992554 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":99,"threads_started":1}
W20260812 18:59:37.992784 8566 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:37.987455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:51231 (local address 127.29.112.254:38329)
0812 18:59:37.987608 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:37.987612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:37.987724 (+ 112us) server_negotiation.cc:407] Connection header received
0812 18:59:37.987857 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:37.987860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:37.987908 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:37.987997 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:37.988408 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.988917 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.989633 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:37.989841 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:37.989910 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:37.992676 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:51231: BlockingRecv error: recv got EOF from 127.29.112.195:51231 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 18:59:38 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:38 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:38.622478 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:38.616438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:41251)
0812 18:59:38.616723 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:38.616736 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:38.616822 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:38.617063 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:38.617065 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:38.617236 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:38.617331 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:38.617336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.618258 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.618261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:38.618829 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:38.618837 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.619317 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.619319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:38.619379 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:38.619894 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:38.619904 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:38.622266 (+ 2362us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
W20260812 18:59:38.622524 8572 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:38.616499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:41251 (local address 127.29.112.254:38329)
0812 18:59:38.616834 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:38.616838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:38.616855 (+ 17us) server_negotiation.cc:407] Connection header received
0812 18:59:38.616905 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:38.616908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:38.616951 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:38.617036 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:38.617436 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.618144 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.618950 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.619209 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.619301 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:38.622400 (+ 3099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:41251: BlockingRecv error: recv got EOF from 127.29.112.193:41251 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":90,"threads_started":1}
Aug 12 18:59:38 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:38 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:38.811403 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:38.805968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:42517)
0812 18:59:38.806241 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:38.806261 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:38.806382 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:38.806670 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:38.806673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:38.806926 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:38.807037 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:38.807043 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.807745 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.807747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:38.808323 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:38.808331 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.808768 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.808771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:38.808841 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:38.809361 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:38.809373 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:38.811253 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":84,"threads_started":1}
W20260812 18:59:38.811478 8572 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:38.806013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:42517 (local address 127.29.112.254:38329)
0812 18:59:38.806159 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:38.806164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:38.806333 (+ 169us) server_negotiation.cc:407] Connection header received
0812 18:59:38.806481 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:38.806485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:38.806535 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:38.806625 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:38.807146 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.807620 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.808428 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.808641 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.808717 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:38.811377 (+ 2660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:42517: BlockingRecv error: recv got EOF from 127.29.112.194:42517 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 18:59:38 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:38 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:38.999432 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:38.993567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:38891)
0812 18:59:38.993945 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:38.993960 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:38.994113 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:38.994397 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:38.994400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:38.994619 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:38.994742 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:38.994750 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.995508 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.995513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:38.996159 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:38.996168 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.996660 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.996663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:38.996747 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:38.997290 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:38.997302 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:38.999241 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":100,"threads_started":1}
W20260812 18:59:38.999477 8572 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:38.993698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:38891 (local address 127.29.112.254:38329)
0812 18:59:38.993896 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:38.993901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:38.994054 (+ 153us) server_negotiation.cc:407] Connection header received
0812 18:59:38.994230 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:38.994233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:38.994287 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:38.994373 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:38.994847 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.995395 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.996296 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:38.996536 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:38.996644 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:38.999372 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:38891: BlockingRecv error: recv got EOF from 127.29.112.195:38891 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 12 18:59:39 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:39 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:39.628590 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:39.623297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:47499)
0812 18:59:39.623609 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:39.623622 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:39.623714 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:39.623937 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:39.623938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:39.624129 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:39.624228 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:39.624233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:39.624943 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:39.624945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:39.625502 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:39.625510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:39.625958 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:39.625961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:39.626025 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:39.626540 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:39.626551 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:39.628393 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":121,"threads_started":1}
W20260812 18:59:39.628636 8578 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:39.623360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:47499 (local address 127.29.112.254:38329)
0812 18:59:39.623633 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:39.623637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:39.623672 (+ 35us) server_negotiation.cc:407] Connection header received
0812 18:59:39.623776 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:39.623778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:39.623818 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:39.623892 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:39.624318 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:39.624852 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:39.625602 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:39.625848 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:39.625921 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:39.628516 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:47499: BlockingRecv error: recv got EOF from 127.29.112.193:47499 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
Aug 12 18:59:39 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:39 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:39.818125 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:39.812261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:36111)
0812 18:59:39.812599 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:39.812613 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:39.812742 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:39.813024 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:39.813027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:39.813249 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:39.813343 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:39.813349 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:39.814164 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:39.814170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:39.814729 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:39.814737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:39.815199 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:39.815203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:39.815289 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:39.815787 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:39.815798 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:39.818008 (+ 2210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":87,"threads_started":1}
W20260812 18:59:39.818269 8578 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:39.812381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:36111 (local address 127.29.112.254:38329)
0812 18:59:39.812546 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:39.812551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:39.812690 (+ 139us) server_negotiation.cc:407] Connection header received
0812 18:59:39.812848 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:39.812852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:39.812906 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:39.813001 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:39.813464 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:39.814055 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:39.814850 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:39.815082 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:39.815193 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:39.818154 (+ 2961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:36111: BlockingRecv error: recv got EOF from 127.29.112.194:36111 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 18:59:40 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:40 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:40.008036 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:40.000586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:58923)
0812 18:59:40.000952 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:40.000968 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:40.001068 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:40.001408 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:40.001410 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:40.001618 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:40.001722 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:40.001728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.002643 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.002646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:40.003759 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:40.003772 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.004634 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.004638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:40.004791 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:40.005737 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:40.005759 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:40.007898 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":155,"threads_started":1}
W20260812 18:59:40.008554 8578 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:40.000812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:58923 (local address 127.29.112.254:38329)
0812 18:59:40.000999 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:40.001003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:40.001022 (+ 19us) server_negotiation.cc:407] Connection header received
0812 18:59:40.001193 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:40.001196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:40.001249 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:40.001335 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:40.001929 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.002487 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.004107 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.004428 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.004575 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:40.008408 (+ 3833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:58923: BlockingRecv error: recv got EOF from 127.29.112.195:58923 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 12 18:59:40 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:40 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:40.635181 8581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:40.629481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:38447)
0812 18:59:40.629772 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:40.629789 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:40.629925 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:40.630234 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:40.630237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:40.630444 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:40.630545 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:40.630551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.631280 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.631283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:40.631931 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:40.631939 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.632351 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.632354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:40.632439 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:40.633064 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:40.633077 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:40.635027 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":85,"threads_started":1}
W20260812 18:59:40.635229 8582 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:40.629576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:38447 (local address 127.29.112.254:38329)
0812 18:59:40.629863 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:40.629868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:40.629882 (+ 14us) server_negotiation.cc:407] Connection header received
0812 18:59:40.630043 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:40.630045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:40.630103 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:40.630183 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:40.630664 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.631181 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.632034 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.632246 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.632318 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:40.635120 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:38447: BlockingRecv error: recv got EOF from 127.29.112.193:38447 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
Aug 12 18:59:40 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:40 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:40.823951 8583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:40.818932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:57811)
0812 18:59:40.819230 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:40.819243 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:40.819340 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:40.819597 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:40.819599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:40.819784 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:40.819878 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:40.819883 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.820579 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.820581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:40.821086 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:40.821092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.821465 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.821467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:40.821518 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:40.822109 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:40.822120 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:40.823812 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":118,"threads_started":1}
W20260812 18:59:40.824137 8582 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:40.819004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:57811 (local address 127.29.112.254:38329)
0812 18:59:40.819169 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:40.819173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:40.819317 (+ 144us) server_negotiation.cc:407] Connection header received
0812 18:59:40.819432 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:40.819434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:40.819478 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:40.819560 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:40.819998 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.820496 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.821185 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:40.821370 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:40.821436 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:40.823983 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:57811: BlockingRecv error: recv got EOF from 127.29.112.194:57811 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 18:59:41 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:41 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:41.014791 8584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:41.009104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:44281)
0812 18:59:41.009462 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:41.009485 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:41.009654 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:41.009967 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:41.009970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:41.010191 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:41.010286 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:41.010298 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.011122 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.011129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:41.011717 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:41.011726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.012208 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.012212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:41.012306 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:41.012825 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:41.012836 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:41.014634 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":89,"threads_started":1}
W20260812 18:59:41.014844 8582 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:41.009219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:44281 (local address 127.29.112.254:38329)
0812 18:59:41.009422 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:41.009428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:41.009584 (+ 156us) server_negotiation.cc:407] Connection header received
0812 18:59:41.009756 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:41.009760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:41.009836 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:41.009933 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:41.010471 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.011003 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.011850 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.012087 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.012206 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:41.014745 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:44281: BlockingRecv error: recv got EOF from 127.29.112.195:44281 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 12 18:59:41 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:41 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:41.641729 8585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:41.636006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:52193)
0812 18:59:41.636312 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:41.636327 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:41.636441 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:41.636704 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:41.636706 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:41.636934 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:41.637045 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:41.637051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.637774 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.637777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:41.638462 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:41.638473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.638907 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.638910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:41.638973 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:41.639518 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:41.639530 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:41.641542 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":86,"threads_started":1}
W20260812 18:59:41.641897 8586 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:41.636092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:52193 (local address 127.29.112.254:38329)
0812 18:59:41.636341 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:41.636345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:41.636403 (+ 58us) server_negotiation.cc:407] Connection header received
0812 18:59:41.636524 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:41.636526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:41.636568 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:41.636640 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:41.637151 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.637652 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.638575 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.638790 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.638862 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:41.641721 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:52193: BlockingRecv error: recv got EOF from 127.29.112.193:52193 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":86,"threads_started":1}
Aug 12 18:59:41 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:41 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:41.830092 8587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:41.824668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:50787)
0812 18:59:41.824996 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:41.825014 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:41.825145 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:41.825404 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:41.825407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:41.825614 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:41.825703 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:41.825708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.826447 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.826450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:41.826948 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:41.826955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.827386 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.827389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:41.827470 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:41.828080 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:41.828091 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:41.829926 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":104,"threads_started":1}
W20260812 18:59:41.830147 8586 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:41.824782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:50787 (local address 127.29.112.254:38329)
0812 18:59:41.824947 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:41.824952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:41.825087 (+ 135us) server_negotiation.cc:407] Connection header received
0812 18:59:41.825239 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:41.825244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:41.825294 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:41.825376 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:41.825845 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.826335 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.827068 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:41.827279 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:41.827381 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:41.830038 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:50787: BlockingRecv error: recv got EOF from 127.29.112.194:50787 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 18:59:42 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:42 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:42.025359 8588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:42.015640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:53047)
0812 18:59:42.015955 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:42.015968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:42.016098 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:42.016362 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:42.016365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:42.016581 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:42.016671 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:42.016677 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.017431 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.017435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:42.018031 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:42.018040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.018534 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.018537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:42.018623 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:42.022662 (+ 4039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:42.022673 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:42.025043 (+ 2370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":88,"threads_started":1}
W20260812 18:59:42.025408 8586 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:42.015732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:53047 (local address 127.29.112.254:38329)
0812 18:59:42.015873 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:42.015886 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:42.016041 (+ 155us) server_negotiation.cc:407] Connection header received
0812 18:59:42.016195 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:42.016199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:42.016252 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:42.016340 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:42.016789 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.017314 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.018145 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.018414 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.018527 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:42.025244 (+ 6717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:53047: BlockingRecv error: recv got EOF from 127.29.112.195:53047 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 18:59:42 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:42 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:42.648299 8589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:42.642586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:56949)
0812 18:59:42.642938 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:42.642954 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:42.643108 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:42.643395 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:42.643397 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:42.643620 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:42.643715 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:42.643722 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.644520 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.644525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:42.645074 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:42.645082 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.645556 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.645560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:42.645642 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:42.646213 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:42.646226 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:42.648195 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":78,"threads_started":1}
W20260812 18:59:42.648540 8591 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:42.642587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:56949 (local address 127.29.112.254:38329)
0812 18:59:42.642891 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:42.642896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:42.643042 (+ 146us) server_negotiation.cc:407] Connection header received
0812 18:59:42.643230 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:42.643233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:42.643287 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:42.643385 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:42.643837 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.644376 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.645197 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.645438 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.645555 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:42.648363 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:56949: BlockingRecv error: recv got EOF from 127.29.112.193:56949 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":84,"threads_started":1}
Aug 12 18:59:42 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:42 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:42.836160 8593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:42.830844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:36757)
0812 18:59:42.831213 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:42.831224 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:42.831338 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:42.831633 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:42.831636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:42.831873 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:42.831981 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:42.831987 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.832730 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.832735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:42.833279 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:42.833286 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.833716 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.833720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:42.833799 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:42.834291 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:42.834313 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:42.836050 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":127,"threads_started":1}
W20260812 18:59:42.836227 8591 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:42.830953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:36757 (local address 127.29.112.254:38329)
0812 18:59:42.831098 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:42.831104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:42.831297 (+ 193us) server_negotiation.cc:407] Connection header received
0812 18:59:42.831452 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:42.831455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:42.831512 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:42.831599 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:42.832121 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.832624 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.833398 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:42.833609 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:42.833713 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:42.836148 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:36757: BlockingRecv error: recv got EOF from 127.29.112.194:36757 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 18:59:43 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:43 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:43.031775 8594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:43.026269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:56465)
0812 18:59:43.026598 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:43.026613 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:43.026744 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:43.027035 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:43.027038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:43.027281 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:43.027411 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:43.027422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.028125 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.028127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:43.028709 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:43.028717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.029116 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.029118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:43.029177 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:43.029652 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:43.029663 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:43.031642 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":98,"threads_started":1}
W20260812 18:59:43.031893 8591 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:43.026344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:56465 (local address 127.29.112.254:38329)
0812 18:59:43.026494 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:43.026499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:43.026704 (+ 205us) server_negotiation.cc:407] Connection header received
0812 18:59:43.026847 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:43.026851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:43.026904 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:43.026976 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:43.027530 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.028038 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.028807 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.029024 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.029097 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:43.031779 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:56465: BlockingRecv error: recv got EOF from 127.29.112.195:56465 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 18:59:43 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:43 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:43.654325 8595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:43.649072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:50599)
0812 18:59:43.649367 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:43.649385 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:43.649494 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:43.649728 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:43.649730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:43.649947 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:43.650039 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:43.650044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.650919 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.650922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:43.651511 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:43.651520 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.651965 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.651967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:43.652055 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:43.652564 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:43.652574 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:43.654191 (+ 1617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":75,"threads_started":1}
W20260812 18:59:43.654505 8596 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:43.649171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:50599 (local address 127.29.112.254:38329)
0812 18:59:43.649515 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:43.649519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:43.649531 (+ 12us) server_negotiation.cc:407] Connection header received
0812 18:59:43.649565 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:43.649567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:43.649607 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:43.649681 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:43.650156 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.650807 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.651609 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.651858 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.651952 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:43.654332 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:50599: BlockingRecv error: recv got EOF from 127.29.112.193:50599 (error 108)
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":85,"threads_started":1}
Aug 12 18:59:43 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:43 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:43.842520 8597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:43.836967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:46095)
0812 18:59:43.837234 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:43.837246 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:43.837329 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:43.837573 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:43.837575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:43.837789 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:43.837943 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:43.837950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.838707 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.838709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:43.839275 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:43.839282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.839745 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.839759 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:43.839820 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:43.840306 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:43.840317 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:43.842347 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":115,"threads_started":1}
W20260812 18:59:43.842617 8596 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:43.837112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:46095 (local address 127.29.112.254:38329)
0812 18:59:43.837324 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:43.837328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:43.837345 (+ 17us) server_negotiation.cc:407] Connection header received
0812 18:59:43.837389 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:43.837391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:43.837435 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:43.837511 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:43.838054 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.838611 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.839401 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:43.839640 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:43.839731 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:43.842489 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:46095: BlockingRecv error: recv got EOF from 127.29.112.194:46095 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 18:59:44 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:44 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.195@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:44.037739 8598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:44.032610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.195:58125)
0812 18:59:44.032875 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:44.032887 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:44.032978 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:44.033190 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:44.033192 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:44.033363 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:44.033455 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:44.033459 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.034121 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.034123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:44.034681 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:44.034688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.035078 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.035081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:44.035137 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:44.035609 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:44.035620 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:44.037621 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":99,"threads_started":1}
W20260812 18:59:44.037940 8596 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:44.032699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:58125 (local address 127.29.112.254:38329)
0812 18:59:44.032836 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:44.032840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:44.032946 (+ 106us) server_negotiation.cc:407] Connection header received
0812 18:59:44.033046 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:44.033049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:44.033090 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:44.033154 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:44.033546 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.034038 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.034786 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.034990 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.035060 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:44.037745 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.195:58125: BlockingRecv error: recv got EOF from 127.29.112.195:58125 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 18:59:44 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:44 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.193@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:44.660315 8599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:44.655219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.193:57609)
0812 18:59:44.655513 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:44.655526 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:44.655607 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:44.655847 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:44.655849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:44.656033 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:44.656127 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:44.656132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.656846 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.656848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:44.657395 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:44.657402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.657801 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.657803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:44.657936 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:44.658426 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:44.658437 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:44.660205 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":109,"threads_started":1}
W20260812 18:59:44.660470 8601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:44.655321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:57609 (local address 127.29.112.254:38329)
0812 18:59:44.655569 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:44.655572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:44.655585 (+ 13us) server_negotiation.cc:407] Connection header received
0812 18:59:44.655693 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:44.655696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:44.655739 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:44.655814 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:44.656224 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.656758 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.657491 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.657706 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.657781 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:44.660332 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.193:57609: BlockingRecv error: recv got EOF from 127.29.112.193:57609 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
Aug 12 18:59:44 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 18:59:44 dist-test-slave-6nmv krb5kdc[7347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.194@KRBTEST.COM for oryx/127.29.112.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 18:59:44.849392 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:44.843463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38329 (local address 127.29.112.194:51923)
0812 18:59:44.843785 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:44.843807 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:44.843957 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:44.844235 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:44.844238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:44.844463 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:44.844576 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:44.844583 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.845379 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.845384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:44.845971 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:44.845980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.846492 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.846496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:44.846600 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:44.847281 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:44.847299 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:44.849281 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.254:38329: Server oryx/127.29.112.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":83,"threads_started":1}
W20260812 18:59:44.849610 8601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:44.843606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:51923 (local address 127.29.112.254:38329)
0812 18:59:44.843764 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:44.843768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:44.843881 (+ 113us) server_negotiation.cc:407] Connection header received
0812 18:59:44.844059 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:44.844062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:44.844117 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:44.844205 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:44.844712 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.845274 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.846082 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:44.846361 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:44.846489 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:44.849467 (+ 2978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.194:51923: BlockingRecv error: recv got EOF from 127.29.112.194:51923 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 18:59:44.914987 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 8040
I20260812 18:59:44.924578 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 8216
I20260812 18:59:44.931932 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 8374
I20260812 18:59:44.939116 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 7961
2026-08-12T18:59:44Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18039 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T18:59:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:44Z Disabled control of system clock
I20260812 18:59:44.972373 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:43943
--webserver_interface=127.29.112.254
--webserver_port=0
--builtin_ntp_servers=127.29.112.212:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:43943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 18:59:45.079713 8606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:45.079954 8606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:45.080001 8606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:45.083683 8606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:45.083727 8606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:45.083741 8606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:45.083755 8606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:45.083770 8606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:45.087946 8606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:43943
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:43943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:45.088776 8606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:45.090039 8606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:45.097515 8611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:45.097525 8612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:45.097566 8606 server_base.cc:1034] running on GCE node
W20260812 18:59:45.097523 8614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:45.098196 8606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:45.099216 8606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:45.100400 8606 hybrid_clock.cc:648] HybridClock initialized: now 1786561185100384 us; error 30 us; skew 500 ppm
I20260812 18:59:45.102001 8606 webserver.cc:460] Webserver started at http://127.29.112.254:44025/ using document root <none> and password file <none>
I20260812 18:59:45.102488 8606 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:45.102569 8606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:45.102769 8606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:45.104429 8606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "c9312370270d41a9bf7af47d3169e5b4"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "92df6475693f2121d6a0a9ad4f4d787e"
server_key_iv: "dddcff31a563f24f9b1ce78faffd8c25"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.104966 8606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "c9312370270d41a9bf7af47d3169e5b4"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "92df6475693f2121d6a0a9ad4f4d787e"
server_key_iv: "dddcff31a563f24f9b1ce78faffd8c25"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.108124 8606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 18:59:45.110248 8620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:45.111223 8606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:45.111343 8606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "c9312370270d41a9bf7af47d3169e5b4"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "92df6475693f2121d6a0a9ad4f4d787e"
server_key_iv: "dddcff31a563f24f9b1ce78faffd8c25"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.111452 8606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:45.125787 8606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:45.126345 8606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:45.126509 8606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:45.132475 8606 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:43943
I20260812 18:59:45.132484 8681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:43943 every 8 connection(s)
I20260812 18:59:45.133277 8606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:45.135975 8682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:45.137696 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 8606
I20260812 18:59:45.137779 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:45.138095 30147 external_mini_cluster.cc:1428] Setting key b8f54e5f43150b0bfc8a838765675254
I20260812 18:59:45.141319 8682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4: Bootstrap starting.
I20260812 18:59:45.143474 8682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:45.144394 8682 log.cc:824] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:45.146611 8682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4: No bootstrap required, opened a new log
I20260812 18:59:45.147290 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:45.139411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57570 (local address 127.29.112.254:43943)
0812 18:59:45.139984 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:45.139998 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:45.140040 (+ 42us) server_negotiation.cc:407] Connection header received
0812 18:59:45.140621 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:45.140636 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:45.140974 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:45.141293 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:45.141677 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.142766 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.143675 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.144399 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.144571 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:45.145377 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:45.145383 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:45.145408 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:45.145891 (+ 483us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:45.145959 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:45.145985 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:45.146082 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:45.146179 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 18:59:45.146436 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":119,"threads_started":1}
I20260812 18:59:45.150533 8682 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9312370270d41a9bf7af47d3169e5b4" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43943 } }
I20260812 18:59:45.150816 8682 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:45.150902 8682 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9312370270d41a9bf7af47d3169e5b4, State: Initialized, Role: FOLLOWER
I20260812 18:59:45.151337 8682 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9312370270d41a9bf7af47d3169e5b4" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43943 } }
I20260812 18:59:45.151481 8682 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:45.151546 8682 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:45.151693 8682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:45.152568 8682 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9312370270d41a9bf7af47d3169e5b4" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43943 } }
I20260812 18:59:45.152900 8682 leader_election.cc:302] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9312370270d41a9bf7af47d3169e5b4; no voters:
I20260812 18:59:45.153192 8682 leader_election.cc:288] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:45.153307 8687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:45.153546 8687 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [term 1 LEADER]: Becoming Leader. State: Replica: c9312370270d41a9bf7af47d3169e5b4, State: Running, Role: LEADER
I20260812 18:59:45.153931 8687 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9312370270d41a9bf7af47d3169e5b4" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43943 } }
I20260812 18:59:45.154405 8682 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:45.155464 8690 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9312370270d41a9bf7af47d3169e5b4. Latest consensus state: current_term: 1 leader_uuid: "c9312370270d41a9bf7af47d3169e5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9312370270d41a9bf7af47d3169e5b4" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43943 } } }
I20260812 18:59:45.155596 8690 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:45.155774 8688 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9312370270d41a9bf7af47d3169e5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9312370270d41a9bf7af47d3169e5b4" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43943 } } }
I20260812 18:59:45.155896 8688 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:45.156250 8698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:45.159242 8698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:45.164805 8698 catalog_manager.cc:1347] Generated new cluster ID: c68246e950b441e28b4ec33a78461614
I20260812 18:59:45.164872 8698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:45.181373 8698 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:45.182348 8698 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:45.189059 8698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9312370270d41a9bf7af47d3169e5b4: Generated new TSK 0
I20260812 18:59:45.189873 8698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 18:59:45.201303 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43943
--builtin_ntp_servers=127.29.112.212:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 18:59:45.317842 8710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:45.318044 8710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:45.318075 8710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:45.321954 8710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:45.321993 8710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:45.322070 8710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:45.326305 8710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:45.327112 8710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:45.328325 8710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:45.337337 8720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:45.337313 8716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:45.337320 8717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:45.337671 8710 server_base.cc:1034] running on GCE node
I20260812 18:59:45.338088 8710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:45.338732 8710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:45.339927 8710 hybrid_clock.cc:648] HybridClock initialized: now 1786561185339900 us; error 26 us; skew 500 ppm
I20260812 18:59:45.341709 8710 webserver.cc:460] Webserver started at http://127.29.112.193:44503/ using document root <none> and password file <none>
I20260812 18:59:45.342279 8710 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:45.342341 8710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:45.342561 8710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:45.344487 8710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "5f7b3340ce8e4666beb8124658555ed0"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "fa5c2142c08bab7d3e1b44e811cf6683"
server_key_iv: "da7f8e5b9d89252cad8c8a18b3546bdd"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.345038 8710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "5f7b3340ce8e4666beb8124658555ed0"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "fa5c2142c08bab7d3e1b44e811cf6683"
server_key_iv: "da7f8e5b9d89252cad8c8a18b3546bdd"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.348608 8710 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 18:59:45.351119 8728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:45.352272 8710 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:45.352375 8710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "5f7b3340ce8e4666beb8124658555ed0"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "fa5c2142c08bab7d3e1b44e811cf6683"
server_key_iv: "da7f8e5b9d89252cad8c8a18b3546bdd"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.352510 8710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:45.365760 8710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:45.366361 8710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:45.366565 8710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:45.367105 8710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:45.367980 8710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:45.368034 8710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:45.368099 8710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:45.368130 8710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:45.376549 8710 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:41243
I20260812 18:59:45.376585 8859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:41243 every 8 connection(s)
I20260812 18:59:45.377350 8710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:45.378438 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 8710
I20260812 18:59:45.378566 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:45.378851 30147 external_mini_cluster.cc:1428] Setting key d0760b68eaa1815714316ec23be54ca9
I20260812 18:59:45.380982 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43943
--builtin_ntp_servers=127.29.112.212:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 18:59:45.386675 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:45.379062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:56567 (local address 127.29.112.254:43943)
0812 18:59:45.379216 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:45.379222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:45.380330 (+ 1108us) server_negotiation.cc:407] Connection header received
0812 18:59:45.382188 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:45.382190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:45.382230 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:45.382295 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:45.383235 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.383724 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.384834 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.385065 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.385204 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:45.386218 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:45.386221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:45.386224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:45.386235 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:45.386258 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:45.386266 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:45.386340 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:45.386494 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 18:59:45.386547 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 18:59:45.387431 8864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:45.379483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43943 (local address 127.29.112.193:56567)
0812 18:59:45.380181 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:45.380231 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:45.381986 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:45.382457 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:45.382467 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:45.382566 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:45.383104 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:45.383116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.383846 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.383849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:45.384711 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:45.384720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.385241 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.385245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:45.385397 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:45.385978 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:45.385980 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:45.386013 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:45.386066 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:45.386342 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:45.386353 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:45.386359 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 18:59:45.386480 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 18:59:45.386712 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":145,"threads_started":1}
I20260812 18:59:45.388676 8860 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43943
I20260812 18:59:45.388885 8860 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:45.389329 8860 heartbeater.cc:511] Master 127.29.112.254:43943 requested a full tablet report, sending...
I20260812 18:59:45.390964 8641 ts_manager.cc:194] Registered new tserver with Master: 5f7b3340ce8e4666beb8124658555ed0 (127.29.112.193:41243)
I20260812 18:59:45.392601 8641 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.193:56567
W20260812 18:59:45.484393 8865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:45.484586 8865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:45.484617 8865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:45.488303 8865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:45.488343 8865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:45.488389 8865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:45.492771 8865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:45.493587 8865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:45.494809 8865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:45.503935 8876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:45.503876 8872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:45.503971 8874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:45.504045 8865 server_base.cc:1034] running on GCE node
I20260812 18:59:45.504539 8865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:45.505119 8865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:45.506273 8865 hybrid_clock.cc:648] HybridClock initialized: now 1786561185506259 us; error 28 us; skew 500 ppm
I20260812 18:59:45.508114 8865 webserver.cc:460] Webserver started at http://127.29.112.194:39829/ using document root <none> and password file <none>
I20260812 18:59:45.508654 8865 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:45.508721 8865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:45.508956 8865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:45.510905 8865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "d4b76a58f8f94cda9dfeb6009b9d9a5a"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "3cac0a94af28106d61f1e15d3c6a9e0b"
server_key_iv: "5e861aadbd802b10f7bc38d0ef49b91a"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.511452 8865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "d4b76a58f8f94cda9dfeb6009b9d9a5a"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "3cac0a94af28106d61f1e15d3c6a9e0b"
server_key_iv: "5e861aadbd802b10f7bc38d0ef49b91a"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.515029 8865 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 18:59:45.517468 8883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:45.518570 8865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:45.518699 8865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "d4b76a58f8f94cda9dfeb6009b9d9a5a"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "3cac0a94af28106d61f1e15d3c6a9e0b"
server_key_iv: "5e861aadbd802b10f7bc38d0ef49b91a"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.518821 8865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:45.528079 8865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:45.528632 8865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:45.528816 8865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:45.529325 8865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:45.530260 8865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:45.530306 8865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:45.530340 8865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:45.530364 8865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:45.538038 8865 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:42813
I20260812 18:59:45.538107 9017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:42813 every 8 connection(s)
I20260812 18:59:45.538777 8865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:45.546571 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:45.540385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:48905 (local address 127.29.112.254:43943)
0812 18:59:45.540539 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:45.540543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:45.541476 (+ 933us) server_negotiation.cc:407] Connection header received
0812 18:59:45.542687 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:45.542690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:45.542731 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:45.542796 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:45.543498 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.544010 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.544988 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.545189 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.545261 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:45.546216 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:45.546218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:45.546220 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:45.546229 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:45.546244 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:45.546250 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:45.546300 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:45.546423 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 18:59:45.546464 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 18:59:45.547086 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 8865
I20260812 18:59:45.547209 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:45.547381 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:45.540771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43943 (local address 127.29.112.194:48905)
0812 18:59:45.541331 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:45.541378 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:45.542525 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:45.542926 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:45.542932 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:45.543015 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:45.543393 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:45.543404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.544123 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.544125 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:45.544893 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:45.544900 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.545310 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.545312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:45.545431 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:45.546012 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:45.546013 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:45.546046 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:45.546099 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:45.546316 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:45.546322 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:45.546326 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 18:59:45.546406 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 18:59:45.546584 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":102,"threads_started":1}
I20260812 18:59:45.547514 30147 external_mini_cluster.cc:1428] Setting key 168620be85023a474bdbcb771640b421
I20260812 18:59:45.548487 9018 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43943
I20260812 18:59:45.548724 9018 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:45.549283 9018 heartbeater.cc:511] Master 127.29.112.254:43943 requested a full tablet report, sending...
I20260812 18:59:45.549561 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43943
--builtin_ntp_servers=127.29.112.212:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 18:59:45.550374 8641 ts_manager.cc:194] Registered new tserver with Master: d4b76a58f8f94cda9dfeb6009b9d9a5a (127.29.112.194:42813)
I20260812 18:59:45.550899 8641 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.194:48905
W20260812 18:59:45.663762 9022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:45.664022 9022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:45.664060 9022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:45.668358 9022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:45.668417 9022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:45.668489 9022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:45.673418 9022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:45.674623 9022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:45.675992 9022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:45.685626 9032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:45.685633 9030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:45.685717 9022 server_base.cc:1034] running on GCE node
W20260812 18:59:45.685671 9029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:45.686231 9022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:45.686746 9022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:45.687940 9022 hybrid_clock.cc:648] HybridClock initialized: now 1786561185687921 us; error 28 us; skew 500 ppm
I20260812 18:59:45.689796 9022 webserver.cc:460] Webserver started at http://127.29.112.195:33883/ using document root <none> and password file <none>
I20260812 18:59:45.690421 9022 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:45.690510 9022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:45.690717 9022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:45.692654 9022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "ce4ab18ef5b54eceba8bf7a8e983d552"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "f225d866f859cb0068a9ceebb697cc82"
server_key_iv: "eef830460eb41d8ba504f8b36b773ded"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.693264 9022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "ce4ab18ef5b54eceba8bf7a8e983d552"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "f225d866f859cb0068a9ceebb697cc82"
server_key_iv: "eef830460eb41d8ba504f8b36b773ded"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.696789 9022 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 18:59:45.698987 9039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:45.700254 9022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:45.700363 9022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "ce4ab18ef5b54eceba8bf7a8e983d552"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "f225d866f859cb0068a9ceebb697cc82"
server_key_iv: "eef830460eb41d8ba504f8b36b773ded"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.700455 9022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:45.714838 9022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:45.715421 9022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:45.715565 9022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:45.716120 9022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:45.716897 9022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:45.716946 9022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:45.716981 9022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:45.717001 9022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:45.725492 9022 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:42001
I20260812 18:59:45.725519 9170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:42001 every 8 connection(s)
I20260812 18:59:45.726380 9022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:59:45.735028 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:45.728008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:36633 (local address 127.29.112.254:43943)
0812 18:59:45.728147 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:45.728152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:45.729177 (+ 1025us) server_negotiation.cc:407] Connection header received
0812 18:59:45.730492 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:45.730495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:45.730545 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:45.730639 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:45.731382 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.731914 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.733267 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.733472 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.733554 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:45.734640 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:45.734643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:45.734645 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:45.734656 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:45.734675 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:45.734683 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:45.734739 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:45.734868 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 18:59:45.734918 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 18:59:45.735791 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:45.728378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43943 (local address 127.29.112.195:36633)
0812 18:59:45.729034 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:45.729075 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:45.730313 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:45.730757 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:45.730763 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:45.730837 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:45.731267 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:45.731278 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.732058 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.732062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:45.733160 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:45.733170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:45.733618 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:45.733622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:45.733752 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:45.734382 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:45.734384 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:45.734448 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:45.734506 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:45.734756 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:45.734762 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:45.734765 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 18:59:45.734852 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 18:59:45.735052 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":111,"threads_started":1}
I20260812 18:59:45.736253 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 9022
I20260812 18:59:45.736361 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:45.736690 30147 external_mini_cluster.cc:1428] Setting key d80ff24cd273e12a4283e4c19cbde6a8
I20260812 18:59:45.737001 9171 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43943
I20260812 18:59:45.737236 9171 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:45.737717 9171 heartbeater.cc:511] Master 127.29.112.254:43943 requested a full tablet report, sending...
I20260812 18:59:45.738792 8641 ts_manager.cc:194] Registered new tserver with Master: ce4ab18ef5b54eceba8bf7a8e983d552 (127.29.112.195:42001)
I20260812 18:59:45.739331 8641 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.195:36633
I20260812 18:59:45.750430 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 18:59:45.753221 8685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:45.752251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57580 (local address 127.29.112.254:43943)
0812 18:59:45.752372 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:45.752378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:45.752591 (+ 213us) server_negotiation.cc:407] Connection header received
0812 18:59:45.752741 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:45.752746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:45.752790 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:45.752886 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:45.753126 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57580: BlockingRecv error: recv got EOF from 127.0.0.1:57580 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 18:59:45.753548 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 8710
I20260812 18:59:45.764530 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 8865
I20260812 18:59:45.774583 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 9022
I20260812 18:59:45.782446 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 8606
2026-08-12T18:59:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (842 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T18:59:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:45Z Disabled control of system clock
I20260812 18:59:45.815083 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:37975
--webserver_interface=127.29.112.254
--webserver_port=0
--builtin_ntp_servers=127.29.112.212:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:37975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 18:59:45.918124 9186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:45.918350 9186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:45.918385 9186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:45.922231 9186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:45.922274 9186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:45.922288 9186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:45.922302 9186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:45.922315 9186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:45.926499 9186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:37975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:37975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:45.927371 9186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:45.928561 9186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 18:59:45.936646 9186 server_base.cc:1034] running on GCE node
W20260812 18:59:45.936574 9192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:45.936620 9194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:45.936574 9191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:45.937439 9186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:45.938498 9186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:45.939679 9186 hybrid_clock.cc:648] HybridClock initialized: now 1786561185939646 us; error 46 us; skew 500 ppm
I20260812 18:59:45.941452 9186 webserver.cc:460] Webserver started at http://127.29.112.254:35619/ using document root <none> and password file <none>
I20260812 18:59:45.942075 9186 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:45.942137 9186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:45.942363 9186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:45.944298 9186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "d8125cf63c114680ad2acf974d49db85"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "f0f7b267151584e37a5a9f059fdf54b5"
server_key_iv: "e48e5b527802244e1ce2e1d2b063bfe4"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.944839 9186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "d8125cf63c114680ad2acf974d49db85"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "f0f7b267151584e37a5a9f059fdf54b5"
server_key_iv: "e48e5b527802244e1ce2e1d2b063bfe4"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.948501 9186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 18:59:45.950872 9202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:45.952054 9186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:45.952190 9186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "d8125cf63c114680ad2acf974d49db85"
format_stamp: "Formatted at 2026-08-12 18:59:45 on dist-test-slave-6nmv"
server_key: "f0f7b267151584e37a5a9f059fdf54b5"
server_key_iv: "e48e5b527802244e1ce2e1d2b063bfe4"
server_key_version: "encryptionkey@0"
I20260812 18:59:45.952299 9186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:45.956755 9186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:45.957320 9186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:45.957506 9186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:45.964205 9186 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:37975
I20260812 18:59:45.964214 9266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:37975 every 8 connection(s)
I20260812 18:59:45.965107 9186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:45.968423 9267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:45.970230 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 9186
I20260812 18:59:45.970350 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:45.970630 30147 external_mini_cluster.cc:1428] Setting key dadd984d3f3faec95070b52fb5f57e9f
I20260812 18:59:45.975091 9267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85: Bootstrap starting.
I20260812 18:59:45.976262 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:45.972111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59164 (local address 127.29.112.254:37975)
0812 18:59:45.972736 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:45.972755 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:45.972789 (+ 34us) server_negotiation.cc:407] Connection header received
0812 18:59:45.973414 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:45.973429 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:45.973764 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:45.974151 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:45.974928 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:45.974933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:45.974944 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:45.974954 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:45.975056 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:45.975083 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:45.975223 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:45.975370 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 18:59:45.975628 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":135,"threads_started":1}
I20260812 18:59:45.977380 9267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:45.978276 9267 log.cc:824] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:45.980077 9267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85: No bootstrap required, opened a new log
I20260812 18:59:45.983597 9267 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8125cf63c114680ad2acf974d49db85" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37975 } }
I20260812 18:59:45.983841 9267 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:45.983919 9267 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8125cf63c114680ad2acf974d49db85, State: Initialized, Role: FOLLOWER
I20260812 18:59:45.984316 9267 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8125cf63c114680ad2acf974d49db85" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37975 } }
I20260812 18:59:45.984486 9267 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:45.984578 9267 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:45.984696 9267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:45.985611 9267 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8125cf63c114680ad2acf974d49db85" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37975 } }
I20260812 18:59:45.985985 9267 leader_election.cc:302] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8125cf63c114680ad2acf974d49db85; no voters:
I20260812 18:59:45.986296 9267 leader_election.cc:288] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:45.986404 9273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:45.986636 9273 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [term 1 LEADER]: Becoming Leader. State: Replica: d8125cf63c114680ad2acf974d49db85, State: Running, Role: LEADER
I20260812 18:59:45.987239 9273 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8125cf63c114680ad2acf974d49db85" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37975 } }
I20260812 18:59:45.987581 9267 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:45.988919 9275 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8125cf63c114680ad2acf974d49db85. Latest consensus state: current_term: 1 leader_uuid: "d8125cf63c114680ad2acf974d49db85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8125cf63c114680ad2acf974d49db85" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37975 } } }
I20260812 18:59:45.988919 9274 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8125cf63c114680ad2acf974d49db85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8125cf63c114680ad2acf974d49db85" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 37975 } } }
I20260812 18:59:45.989073 9275 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:45.989073 9274 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85 [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:45.989869 9284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:45.992565 9284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:45.997377 9284 catalog_manager.cc:1347] Generated new cluster ID: c56d24516f2f496f858c03dd97af5142
I20260812 18:59:45.997437 9284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:46.016237 9284 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:46.016528 9284 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:46.021080 9284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8125cf63c114680ad2acf974d49db85: Generated new TSK 0
I20260812 18:59:46.021735 9284 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 18:59:46.034310 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37975
--builtin_ntp_servers=127.29.112.212:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 18:59:46.142546 9292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:46.142789 9292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:46.142822 9292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:46.146246 9292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:46.146286 9292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:46.146330 9292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:46.150549 9292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:46.151335 9292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:46.152561 9292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:46.161289 9305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:46.161305 9300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:46.161334 9298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:46.161671 9292 server_base.cc:1034] running on GCE node
I20260812 18:59:46.162056 9292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:46.162604 9292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:46.163784 9292 hybrid_clock.cc:648] HybridClock initialized: now 1786561186163775 us; error 21 us; skew 500 ppm
I20260812 18:59:46.165462 9292 webserver.cc:460] Webserver started at http://127.29.112.193:41083/ using document root <none> and password file <none>
I20260812 18:59:46.166034 9292 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:46.166122 9292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:46.166324 9292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:46.168164 9292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "09f01e0bf17a40139e3f9feae238eb58"
format_stamp: "Formatted at 2026-08-12 18:59:46 on dist-test-slave-6nmv"
server_key: "57946099f800ee945899511bc92a58d0"
server_key_iv: "06ee57e9518b17e6e13fbf0312c1eb70"
server_key_version: "encryptionkey@0"
I20260812 18:59:46.168690 9292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "09f01e0bf17a40139e3f9feae238eb58"
format_stamp: "Formatted at 2026-08-12 18:59:46 on dist-test-slave-6nmv"
server_key: "57946099f800ee945899511bc92a58d0"
server_key_iv: "06ee57e9518b17e6e13fbf0312c1eb70"
server_key_version: "encryptionkey@0"
I20260812 18:59:46.172106 9292 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 18:59:46.174405 9312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:46.175442 9292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:46.175571 9292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "09f01e0bf17a40139e3f9feae238eb58"
format_stamp: "Formatted at 2026-08-12 18:59:46 on dist-test-slave-6nmv"
server_key: "57946099f800ee945899511bc92a58d0"
server_key_iv: "06ee57e9518b17e6e13fbf0312c1eb70"
server_key_version: "encryptionkey@0"
I20260812 18:59:46.175668 9292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:46.179657 9292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:46.180214 9292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:46.180397 9292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:46.180898 9292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:46.181706 9292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:46.181756 9292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:46.181859 9292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:46.181901 9292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:46.190946 9292 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:41173
I20260812 18:59:46.190969 9445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:41173 every 8 connection(s)
I20260812 18:59:46.191792 9292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:46.197193 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:46.193450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:46173 (local address 127.29.112.254:37975)
0812 18:59:46.193607 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:46.193612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:46.194605 (+ 993us) server_negotiation.cc:407] Connection header received
0812 18:59:46.195743 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:46.195745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:46.195779 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:46.195859 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:46.196852 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:46.196854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:46.196856 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:46.196864 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:46.196878 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:46.196885 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:46.196950 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:46.197064 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 18:59:46.197096 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 18:59:46.198031 9448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:46.193861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:37975 (local address 127.29.112.193:46173)
0812 18:59:46.194468 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:46.194505 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:46.195557 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:46.195995 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:46.196000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:46.196075 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:46.196635 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:46.196637 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:46.196674 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:46.196736 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:46.196952 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:46.196958 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:46.196962 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 18:59:46.197084 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 18:59:46.197251 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":107,"threads_started":1}
I20260812 18:59:46.199101 9446 heartbeater.cc:347] Connected to a master server at 127.29.112.254:37975
I20260812 18:59:46.199312 9446 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:46.199692 9446 heartbeater.cc:511] Master 127.29.112.254:37975 requested a full tablet report, sending...
I20260812 18:59:46.200770 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 9292
I20260812 18:59:46.200907 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:46.201186 30147 external_mini_cluster.cc:1428] Setting key 7dbe4ab3d22ac4be72b37b31e30072fa
I20260812 18:59:46.201218 9225 ts_manager.cc:194] Registered new tserver with Master: 09f01e0bf17a40139e3f9feae238eb58 (127.29.112.193:41173)
I20260812 18:59:46.203145 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37975
--builtin_ntp_servers=127.29.112.212:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 18:59:46.310750 9449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:46.310992 9449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:46.311069 9449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:46.314831 9449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:46.314898 9449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:46.314986 9449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:46.319432 9449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:46.320370 9449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:46.321585 9449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:46.330258 9457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:46.330267 9461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:46.330267 9458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:46.330578 9449 server_base.cc:1034] running on GCE node
I20260812 18:59:46.330963 9449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:46.331542 9449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:46.332737 9449 hybrid_clock.cc:648] HybridClock initialized: now 1786561186332707 us; error 43 us; skew 500 ppm
I20260812 18:59:46.334435 9449 webserver.cc:460] Webserver started at http://127.29.112.194:38221/ using document root <none> and password file <none>
I20260812 18:59:46.334940 9449 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:46.335031 9449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:46.335237 9449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:46.336997 9449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "5cf99f28d0384276b8f1505718f1deda"
format_stamp: "Formatted at 2026-08-12 18:59:46 on dist-test-slave-6nmv"
server_key: "14188c1b5926097b442bd55033e3acda"
server_key_iv: "b56ae617c850ecc20020aaa54477f7d1"
server_key_version: "encryptionkey@0"
I20260812 18:59:46.337512 9449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "5cf99f28d0384276b8f1505718f1deda"
format_stamp: "Formatted at 2026-08-12 18:59:46 on dist-test-slave-6nmv"
server_key: "14188c1b5926097b442bd55033e3acda"
server_key_iv: "b56ae617c850ecc20020aaa54477f7d1"
server_key_version: "encryptionkey@0"
I20260812 18:59:46.341090 9449 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 18:59:46.343596 9468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:46.344712 9449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 18:59:46.344823 9449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "5cf99f28d0384276b8f1505718f1deda"
format_stamp: "Formatted at 2026-08-12 18:59:46 on dist-test-slave-6nmv"
server_key: "14188c1b5926097b442bd55033e3acda"
server_key_iv: "b56ae617c850ecc20020aaa54477f7d1"
server_key_version: "encryptionkey@0"
I20260812 18:59:46.344920 9449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:46.349102 9449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:46.349671 9449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:46.349902 9449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:46.350440 9449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:46.351230 9449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:46.351270 9449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:46.351327 9449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:46.351361 9449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:46.360107 9449 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:45503
I20260812 18:59:46.360196 9600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:45503 every 8 connection(s)
I20260812 18:59:46.360992 9449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:46.366783 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:46.362969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:43727 (local address 127.29.112.254:37975)
0812 18:59:46.363137 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:46.363142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:46.364137 (+ 995us) server_negotiation.cc:407] Connection header received
0812 18:59:46.365320 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:46.365322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:46.365354 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:46.365423 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:46.366416 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:46.366419 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:46.366421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:46.366432 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:46.366449 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:46.366457 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:46.366532 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:46.366645 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 18:59:46.366679 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 18:59:46.367547 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:46.363346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:37975 (local address 127.29.112.194:43727)
0812 18:59:46.363996 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:46.364044 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:46.365157 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:46.365562 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:46.365568 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:46.365661 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:46.366172 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:46.366174 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:46.366209 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:46.366269 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:46.366531 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:46.366537 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:46.366541 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 18:59:46.366661 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 18:59:46.366827 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":114,"threads_started":1}
I20260812 18:59:46.368562 9601 heartbeater.cc:347] Connected to a master server at 127.29.112.254:37975
I20260812 18:59:46.368758 9601 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:46.369179 9601 heartbeater.cc:511] Master 127.29.112.254:37975 requested a full tablet report, sending...
I20260812 18:59:46.369354 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 9449
I20260812 18:59:46.369465 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:46.369753 30147 external_mini_cluster.cc:1428] Setting key 3e32a631730c23516e01ff7a19c986f0
I20260812 18:59:46.370260 9224 ts_manager.cc:194] Registered new tserver with Master: 5cf99f28d0384276b8f1505718f1deda (127.29.112.194:45503)
I20260812 18:59:46.371560 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37975
--builtin_ntp_servers=127.29.112.212:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 18:59:46.475505 9609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:46.475713 9609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:46.475746 9609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:46.479789 9609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:46.479835 9609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:46.479918 9609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:46.484372 9609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:46.485391 9609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:46.486704 9609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:46.496140 9618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:46.496171 9615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:46.496172 9614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:46.496276 9609 server_base.cc:1034] running on GCE node
I20260812 18:59:46.496843 9609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:46.497424 9609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:46.498610 9609 hybrid_clock.cc:648] HybridClock initialized: now 1786561186498601 us; error 26 us; skew 500 ppm
I20260812 18:59:46.500288 9609 webserver.cc:460] Webserver started at http://127.29.112.195:46065/ using document root <none> and password file <none>
I20260812 18:59:46.500838 9609 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:46.500927 9609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:46.501160 9609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:46.502978 9609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "f458314f6c25469f9ba517f5d11c02b6"
format_stamp: "Formatted at 2026-08-12 18:59:46 on dist-test-slave-6nmv"
server_key: "5064d16c6a869d257d72592ee42dc435"
server_key_iv: "26fc3f19378d44faee75ab8e9c8a2e5e"
server_key_version: "encryptionkey@0"
I20260812 18:59:46.503538 9609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "f458314f6c25469f9ba517f5d11c02b6"
format_stamp: "Formatted at 2026-08-12 18:59:46 on dist-test-slave-6nmv"
server_key: "5064d16c6a869d257d72592ee42dc435"
server_key_iv: "26fc3f19378d44faee75ab8e9c8a2e5e"
server_key_version: "encryptionkey@0"
I20260812 18:59:46.506739 9609 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 18:59:46.509057 9624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:46.510236 9609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:46.510354 9609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "f458314f6c25469f9ba517f5d11c02b6"
format_stamp: "Formatted at 2026-08-12 18:59:46 on dist-test-slave-6nmv"
server_key: "5064d16c6a869d257d72592ee42dc435"
server_key_iv: "26fc3f19378d44faee75ab8e9c8a2e5e"
server_key_version: "encryptionkey@0"
I20260812 18:59:46.510453 9609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:46.514541 9609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:46.515072 9609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:46.515241 9609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:46.515799 9609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:46.516597 9609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:46.516664 9609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:46.516721 9609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:46.516768 9609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:46.525481 9609 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:34973
I20260812 18:59:46.525511 9761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:34973 every 8 connection(s)
I20260812 18:59:46.526407 9609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:59:46.527452 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 9609
I20260812 18:59:46.527566 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:46.527819 30147 external_mini_cluster.cc:1428] Setting key 7a4efb4640acb70f57587304ce07ee1f
I20260812 18:59:46.532409 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:46.528224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:57023 (local address 127.29.112.254:37975)
0812 18:59:46.528351 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:46.528357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:46.529590 (+ 1233us) server_negotiation.cc:407] Connection header received
0812 18:59:46.530872 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:46.530875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:46.530912 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:46.530978 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:46.532034 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:46.532037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:46.532039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 18:59:46.532050 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:46.532070 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 18:59:46.532078 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:46.532153 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:46.532273 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 18:59:46.532308 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 18:59:46.533188 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:46.528696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:37975 (local address 127.29.112.195:57023)
0812 18:59:46.529436 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:46.529484 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:46.530699 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:46.531130 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:46.531136 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:46.531216 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:46.531794 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 18:59:46.531796 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:46.531833 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 18:59:46.531898 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:46.532155 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:46.532162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:46.532166 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 18:59:46.532287 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 18:59:46.532458 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":135,"threads_started":1}
I20260812 18:59:46.534207 9762 heartbeater.cc:347] Connected to a master server at 127.29.112.254:37975
I20260812 18:59:46.534412 9762 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:46.534825 9762 heartbeater.cc:511] Master 127.29.112.254:37975 requested a full tablet report, sending...
I20260812 18:59:46.535759 9225 ts_manager.cc:194] Registered new tserver with Master: f458314f6c25469f9ba517f5d11c02b6 (127.29.112.195:34973)
I20260812 18:59:46.541505 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 18:59:46.544201 9271 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 18:59:46.543498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59180 (local address 127.29.112.254:37975)
0812 18:59:46.543608 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:46.543611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:46.543782 (+ 171us) server_negotiation.cc:407] Connection header received
0812 18:59:46.543871 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:46.543873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:46.543901 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:46.543962 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:46.544115 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59180: BlockingRecv error: recv got EOF from 127.0.0.1:59180 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 18:59:46.544512 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 9292
I20260812 18:59:46.557185 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 9449
I20260812 18:59:46.564157 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 9609
I20260812 18:59:46.575399 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 9186
2026-08-12T18:59:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (791 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:59:46 dist-test-slave-6nmv krb5kdc[9775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:59:46 dist-test-slave-6nmv krb5kdc[9775](info): set up 2 sockets
Aug 12 18:59:46 dist-test-slave-6nmv krb5kdc[9775](info): commencing operation
krb5kdc: starting...
W20260812 18:59:47.661074 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:47 dist-test-slave-6nmv krb5kdc[9775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 18:59:47.774703 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:33607
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:34341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:33607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:47.875581 9793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:47.875803 9793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:47.875841 9793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:47.879957 9793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 18:59:47.880012 9793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:47.880033 9793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:47.880084 9793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 18:59:47.880134 9793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 18:59:47.885105 9793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:33607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:33607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:47.886027 9793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:47.886905 9793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:47.895071 9798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:47.895067 9801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:47.895073 9799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:47.895316 9793 server_base.cc:1034] running on GCE node
I20260812 18:59:47.895982 9793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:47.896950 9793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:47.898105 9793 hybrid_clock.cc:648] HybridClock initialized: now 1786561187898075 us; error 41 us; skew 500 ppm
Aug 12 18:59:47 dist-test-slave-6nmv krb5kdc[9775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561187, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:47.903786 9793 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 18:59:47.904613 9793 webserver.cc:460] Webserver started at http://127.29.112.254:37331/ using document root <none> and password file <none>
I20260812 18:59:47.905139 9793 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:47.905262 9793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:47.905498 9793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:47.907425 9793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "5b96f82535a84a199897f824917334cf"
format_stamp: "Formatted at 2026-08-12 18:59:47 on dist-test-slave-6nmv"
server_key: "fb2f78441e8cdf016c693a85e49ab1f3"
server_key_iv: "e506d02d78e221a42ca214d1666e9954"
server_key_version: "encryptionkey@0"
I20260812 18:59:47.907965 9793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "5b96f82535a84a199897f824917334cf"
format_stamp: "Formatted at 2026-08-12 18:59:47 on dist-test-slave-6nmv"
server_key: "fb2f78441e8cdf016c693a85e49ab1f3"
server_key_iv: "e506d02d78e221a42ca214d1666e9954"
server_key_version: "encryptionkey@0"
I20260812 18:59:47.911470 9793 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 18:59:47.913673 9808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:47.914847 9793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:47.914937 9793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "5b96f82535a84a199897f824917334cf"
format_stamp: "Formatted at 2026-08-12 18:59:47 on dist-test-slave-6nmv"
server_key: "fb2f78441e8cdf016c693a85e49ab1f3"
server_key_iv: "e506d02d78e221a42ca214d1666e9954"
server_key_version: "encryptionkey@0"
I20260812 18:59:47.915060 9793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:47.934320 9793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:47.936859 9793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:47.937059 9793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:47.943602 9793 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:33607
I20260812 18:59:47.943673 9873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:33607 every 8 connection(s)
I20260812 18:59:47.944499 9793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 18:59:47.946935 9874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:47.949796 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 9793
I20260812 18:59:47.949942 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 18:59:47.950239 30147 external_mini_cluster.cc:1428] Setting key d105526e34a6f52b464310afceb09bd9
I20260812 18:59:47.952472 9874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf: Bootstrap starting.
I20260812 18:59:47.954981 9874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:47.955868 9874 log.cc:824] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:47.958003 9874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf: No bootstrap required, opened a new log
Aug 12 18:59:47 dist-test-slave-6nmv krb5kdc[9775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:47.961263 9874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96f82535a84a199897f824917334cf" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33607 } }
I20260812 18:59:47.961469 9874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:47.961550 9874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b96f82535a84a199897f824917334cf, State: Initialized, Role: FOLLOWER
I20260812 18:59:47.961984 9874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96f82535a84a199897f824917334cf" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33607 } }
I20260812 18:59:47.962142 9874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 18:59:47.962227 9874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 18:59:47.962320 9874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:47.963163 9874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96f82535a84a199897f824917334cf" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33607 } }
I20260812 18:59:47.963533 9874 leader_election.cc:302] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b96f82535a84a199897f824917334cf; no voters:
I20260812 18:59:47.963855 9874 leader_election.cc:288] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 18:59:47.963917 9879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:47.964133 9879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [term 1 LEADER]: Becoming Leader. State: Replica: 5b96f82535a84a199897f824917334cf, State: Running, Role: LEADER
I20260812 18:59:47.964416 9877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:47.951536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.29.112.254:33607)
0812 18:59:47.951898 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:47.951912 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:47.951943 (+ 31us) server_negotiation.cc:407] Connection header received
0812 18:59:47.952573 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:47.952595 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:47.952903 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:47.953156 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:47.954061 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:47.955163 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:47.956280 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:47.956877 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:47.956992 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:47.959679 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:47.959684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:47.959696 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:47.959720 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:47.961300 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:47.961754 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:47.961759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:47.961762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:47.961920 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:47.962258 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:47.962261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:47.962262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:47.962676 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:47.962879 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:47.963177 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 18:59:47.963503 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":113,"threads_started":1}
I20260812 18:59:47.964486 9879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96f82535a84a199897f824917334cf" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33607 } }
I20260812 18:59:47.965160 9874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 18:59:47.966361 9880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b96f82535a84a199897f824917334cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96f82535a84a199897f824917334cf" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33607 } } }
I20260812 18:59:47.966490 9880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:47.966820 9881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b96f82535a84a199897f824917334cf. Latest consensus state: current_term: 1 leader_uuid: "5b96f82535a84a199897f824917334cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b96f82535a84a199897f824917334cf" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33607 } } }
I20260812 18:59:47.967025 9881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf [sys.catalog]: This master's current role is: LEADER
I20260812 18:59:47.967389 9889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 18:59:47.970021 9889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 18:59:47.975167 9889 catalog_manager.cc:1347] Generated new cluster ID: 3f22f545576d4355af556e695a500aaa
I20260812 18:59:47.975230 9889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 18:59:48.007439 9889 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 18:59:48.008482 9889 catalog_manager.cc:1504] Loading token signing keys...
I20260812 18:59:48.017057 9889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf: Generated new TSK 0
I20260812 18:59:48.017710 9889 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 18:59:48.046623 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33607
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:34341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:48.149205 9905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:48.149420 9905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:48.149459 9905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:48.153203 9905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:48.153254 9905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:48.153307 9905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 18:59:48.158035 9905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:48.158881 9905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:48.159737 9905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:48.168815 9915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:48.168819 9911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:48.168831 9913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:48.168947 9905 server_base.cc:1034] running on GCE node
I20260812 18:59:48.169426 9905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:48.170017 9905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:48.171171 9905 hybrid_clock.cc:648] HybridClock initialized: now 1786561188171166 us; error 39 us; skew 500 ppm
Aug 12 18:59:48 dist-test-slave-6nmv krb5kdc[9775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:48.173794 9905 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 18:59:48.174588 9905 webserver.cc:460] Webserver started at http://127.29.112.193:40085/ using document root <none> and password file <none>
I20260812 18:59:48.175110 9905 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:48.175199 9905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:48.175410 9905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:48.177188 9905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "3a594eaacedd4f61a84df1e171fb3965"
format_stamp: "Formatted at 2026-08-12 18:59:48 on dist-test-slave-6nmv"
server_key: "57bbfcbd5045f037231049750a8e5a03"
server_key_iv: "9bbf3411ffb529c5734dec1b523d7407"
server_key_version: "encryptionkey@0"
I20260812 18:59:48.177692 9905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "3a594eaacedd4f61a84df1e171fb3965"
format_stamp: "Formatted at 2026-08-12 18:59:48 on dist-test-slave-6nmv"
server_key: "57bbfcbd5045f037231049750a8e5a03"
server_key_iv: "9bbf3411ffb529c5734dec1b523d7407"
server_key_version: "encryptionkey@0"
I20260812 18:59:48.181216 9905 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 18:59:48.183388 9924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:48.184417 9905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 18:59:48.184541 9905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "3a594eaacedd4f61a84df1e171fb3965"
format_stamp: "Formatted at 2026-08-12 18:59:48 on dist-test-slave-6nmv"
server_key: "57bbfcbd5045f037231049750a8e5a03"
server_key_iv: "9bbf3411ffb529c5734dec1b523d7407"
server_key_version: "encryptionkey@0"
I20260812 18:59:48.184636 9905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:48.196583 9905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:48.199177 9905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:48.199360 9905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:48.199862 9905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:48.200645 9905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:48.200716 9905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:48.200780 9905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:48.200829 9905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:48.209425 9905 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:33477
I20260812 18:59:48.209460 10051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:33477 every 8 connection(s)
I20260812 18:59:48.210314 9905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 18:59:48.212066 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 9905
I20260812 18:59:48.212180 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 18:59:48.212440 30147 external_mini_cluster.cc:1428] Setting key 7d91d6977a6fda1d093a635f20a47029
Aug 12 18:59:48 dist-test-slave-6nmv krb5kdc[9775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 18:59:48.224151 9877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.212022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:55247 (local address 127.29.112.254:33607)
0812 18:59:48.212185 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:48.212191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:48.213050 (+ 859us) server_negotiation.cc:407] Connection header received
0812 18:59:48.214493 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:48.214499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:48.214556 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:48.214649 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:48.215733 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.216307 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.217526 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.217773 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.217889 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.220640 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:48.220643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:48.220645 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:48.220665 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:48.222183 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:48.222680 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:48.222688 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:48.222689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:48.222760 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:48.223085 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:48.223090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:48.223091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:48.223291 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:48.223412 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:48.223847 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 18:59:48.223976 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 18:59:48.224869 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.212377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33607 (local address 127.29.112.193:55247)
0812 18:59:48.212903 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:48.212951 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:48.214281 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:48.214786 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:48.214792 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:48.215114 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:48.215555 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.215567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.216434 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.216439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.217410 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.217419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.217947 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.217962 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.218185 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.218770 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:48.218794 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:48.220469 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:48.222298 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:48.222304 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:48.222316 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:48.222570 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:48.222849 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:48.222853 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:48.222855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:48.222989 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:48.223411 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:48.223417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:48.223638 (+ 221us) client_negotiation.cc:769] Sending connection context
0812 18:59:48.223809 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 18:59:48.224105 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":153,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
I20260812 18:59:48.226197 10052 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33607
I20260812 18:59:48.226415 10052 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:48.226991 10052 heartbeater.cc:511] Master 127.29.112.254:33607 requested a full tablet report, sending...
I20260812 18:59:48.228457 9829 ts_manager.cc:194] Registered new tserver with Master: 3a594eaacedd4f61a84df1e171fb3965 (127.29.112.193:33477)
I20260812 18:59:48.230013 9829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:55247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 18:59:48.240564 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33607
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:34341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:48.340893 10060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:48.341105 10060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:48.341142 10060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:48.344643 10060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:48.344689 10060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:48.344769 10060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 18:59:48.349303 10060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:48.350106 10060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:48.350875 10060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:48.359485 10068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:48.359498 10065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:48.359500 10066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:48.359573 10060 server_base.cc:1034] running on GCE node
I20260812 18:59:48.360134 10060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:48.360635 10060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:48.361783 10060 hybrid_clock.cc:648] HybridClock initialized: now 1786561188361768 us; error 26 us; skew 500 ppm
Aug 12 18:59:48 dist-test-slave-6nmv krb5kdc[9775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:48.364436 10060 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 18:59:48.365387 10060 webserver.cc:460] Webserver started at http://127.29.112.194:41725/ using document root <none> and password file <none>
I20260812 18:59:48.365988 10060 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:48.366086 10060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:48.366298 10060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:48.368180 10060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "f4e1e69e49ec4cef98d24be00926bf06"
format_stamp: "Formatted at 2026-08-12 18:59:48 on dist-test-slave-6nmv"
server_key: "ce935489ca7228d5a1d91ac832776993"
server_key_iv: "8d3461f88b71c3a1fdc30e3483f29fc6"
server_key_version: "encryptionkey@0"
I20260812 18:59:48.368692 10060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "f4e1e69e49ec4cef98d24be00926bf06"
format_stamp: "Formatted at 2026-08-12 18:59:48 on dist-test-slave-6nmv"
server_key: "ce935489ca7228d5a1d91ac832776993"
server_key_iv: "8d3461f88b71c3a1fdc30e3483f29fc6"
server_key_version: "encryptionkey@0"
I20260812 18:59:48.372093 10060 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 18:59:48.374385 10077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:48.375485 10060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 18:59:48.375625 10060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "f4e1e69e49ec4cef98d24be00926bf06"
format_stamp: "Formatted at 2026-08-12 18:59:48 on dist-test-slave-6nmv"
server_key: "ce935489ca7228d5a1d91ac832776993"
server_key_iv: "8d3461f88b71c3a1fdc30e3483f29fc6"
server_key_version: "encryptionkey@0"
I20260812 18:59:48.375715 10060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:48.394920 10060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:48.397915 10060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:48.398125 10060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:48.398679 10060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:48.399454 10060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:48.399508 10060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:48.399581 10060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:48.399619 10060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:48.408428 10060 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:32895
I20260812 18:59:48.408455 10205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:32895 every 8 connection(s)
I20260812 18:59:48.409245 10060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 18:59:48.416170 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 10060
I20260812 18:59:48.416271 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 18:59:48.416579 30147 external_mini_cluster.cc:1428] Setting key e4b97ea3e05802ff8bf330e2185d43b9
Aug 12 18:59:48 dist-test-slave-6nmv krb5kdc[9775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:48.422035 9877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.410972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:59641 (local address 127.29.112.254:33607)
0812 18:59:48.411096 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:48.411101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:48.411875 (+ 774us) server_negotiation.cc:407] Connection header received
0812 18:59:48.413153 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:48.413155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:48.413202 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:48.413271 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:48.414462 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.414957 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.415877 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.416067 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.416136 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.418683 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:48.418686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:48.418689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:48.418719 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:48.420220 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:48.420692 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:48.420697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:48.420698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:48.420772 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:48.421082 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:48.421086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:48.421087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:48.421248 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:48.421345 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:48.421753 (+ 408us) server_negotiation.cc:299] Negotiation successful
0812 18:59:48.421887 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 18:59:48.422784 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.411262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33607 (local address 127.29.112.194:59641)
0812 18:59:48.411713 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:48.411754 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:48.412950 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:48.413491 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:48.413498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:48.413894 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:48.414325 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.414337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.415055 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.415058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.415778 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.415786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.416206 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.416211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.416387 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.416903 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:48.416920 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:48.418531 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:48.420341 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:48.420346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:48.420357 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:48.420596 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:48.420863 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:48.420866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:48.420867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:48.420995 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:48.421345 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:48.421352 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:48.421554 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 18:59:48.421713 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 18:59:48.422008 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260812 18:59:48.423832 10206 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33607
I20260812 18:59:48.424052 10206 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:48.424463 10206 heartbeater.cc:511] Master 127.29.112.254:33607 requested a full tablet report, sending...
I20260812 18:59:48.425419 9829 ts_manager.cc:194] Registered new tserver with Master: f4e1e69e49ec4cef98d24be00926bf06 (127.29.112.194:32895)
I20260812 18:59:48.426025 9829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:59641
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 18:59:48.444614 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33607
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:34341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 18:59:48.546835 10214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 18:59:48.547055 10214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 18:59:48.547096 10214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 18:59:48.550931 10214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 18:59:48.550972 10214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 18:59:48.551023 10214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 18:59:48.555788 10214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 18:59:48.556674 10214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 18:59:48.557562 10214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 18:59:48.566565 10222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:48.566615 10220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 18:59:48.566826 10219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 18:59:48.566892 10214 server_base.cc:1034] running on GCE node
I20260812 18:59:48.567214 10214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 18:59:48.567813 10214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 18:59:48.568985 10214 hybrid_clock.cc:648] HybridClock initialized: now 1786561188568980 us; error 29 us; skew 500 ppm
Aug 12 18:59:48 dist-test-slave-6nmv krb5kdc[9775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 18:59:48.571678 10214 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 18:59:48.572525 10214 webserver.cc:460] Webserver started at http://127.29.112.195:46623/ using document root <none> and password file <none>
I20260812 18:59:48.573167 10214 fs_manager.cc:359] Metadata directory not provided
I20260812 18:59:48.573267 10214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 18:59:48.573501 10214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 18:59:48.575450 10214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "aaadb68ee8e04ce19210893d90afdcc9"
format_stamp: "Formatted at 2026-08-12 18:59:48 on dist-test-slave-6nmv"
server_key: "7e49ff3f4975f04042ca7ce2f35956f7"
server_key_iv: "5941782d4c9dfcc15cd97ac796b4d7f7"
server_key_version: "encryptionkey@0"
I20260812 18:59:48.575991 10214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "aaadb68ee8e04ce19210893d90afdcc9"
format_stamp: "Formatted at 2026-08-12 18:59:48 on dist-test-slave-6nmv"
server_key: "7e49ff3f4975f04042ca7ce2f35956f7"
server_key_iv: "5941782d4c9dfcc15cd97ac796b4d7f7"
server_key_version: "encryptionkey@0"
I20260812 18:59:48.579648 10214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 18:59:48.582090 10232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:48.583247 10214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 18:59:48.583389 10214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "aaadb68ee8e04ce19210893d90afdcc9"
format_stamp: "Formatted at 2026-08-12 18:59:48 on dist-test-slave-6nmv"
server_key: "7e49ff3f4975f04042ca7ce2f35956f7"
server_key_iv: "5941782d4c9dfcc15cd97ac796b4d7f7"
server_key_version: "encryptionkey@0"
I20260812 18:59:48.583499 10214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 18:59:48.597198 10214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 18:59:48.600381 10214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 18:59:48.600558 10214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 18:59:48.601264 10214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 18:59:48.602360 10214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 18:59:48.602412 10214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:48.602464 10214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 18:59:48.602494 10214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 18:59:48.612214 10214 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:36189
I20260812 18:59:48.612243 10368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:36189 every 8 connection(s)
I20260812 18:59:48.613039 10214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 18:59:48.619933 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 10214
I20260812 18:59:48.620062 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 18:59:48.620344 30147 external_mini_cluster.cc:1428] Setting key 5463d515635fda6a68e056c8d9737cdd
Aug 12 18:59:48 dist-test-slave-6nmv krb5kdc[9775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561188, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 18:59:48.625206 9877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.614625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:39155 (local address 127.29.112.254:33607)
0812 18:59:48.614749 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:48.614753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:48.615429 (+ 676us) server_negotiation.cc:407] Connection header received
0812 18:59:48.616695 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:48.616698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:48.616755 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:48.616848 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:48.618025 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.618545 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.619743 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.619994 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.620102 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.622559 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:48.622562 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:48.622564 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:48.622587 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:48.623654 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:48.624060 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:48.624062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:48.624063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:48.624130 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:48.624405 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:48.624407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:48.624408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:48.624567 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:48.624639 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:48.625011 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 18:59:48.625100 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 18:59:48.625965 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.614915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33607 (local address 127.29.112.195:39155)
0812 18:59:48.615292 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:48.615326 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:48.616495 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:48.617021 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:48.617027 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:48.617325 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0812 18:59:48.617889 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.617905 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.618690 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.618693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.619617 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.619627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.620135 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.620138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.620317 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.620824 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 18:59:48.620848 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 18:59:48.622416 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 18:59:48.623747 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:48.623751 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:48.623760 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:48.623980 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:48.624203 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 18:59:48.624204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 18:59:48.624206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 18:59:48.624320 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 18:59:48.624648 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 18:59:48.624651 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 18:59:48.624834 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 18:59:48.624965 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 18:59:48.625173 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":96,"threads_started":1}
I20260812 18:59:48.626848 10369 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33607
I20260812 18:59:48.627063 10369 heartbeater.cc:464] Registering TS with master...
I20260812 18:59:48.627470 10369 heartbeater.cc:511] Master 127.29.112.254:33607 requested a full tablet report, sending...
I20260812 18:59:48.628333 9829 ts_manager.cc:194] Registered new tserver with Master: aaadb68ee8e04ce19210893d90afdcc9 (127.29.112.195:36189)
I20260812 18:59:48.628890 9829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:39155
I20260812 18:59:48.633679 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 18:59:48.641346 9877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.635351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59084 (local address 127.29.112.254:33607)
0812 18:59:48.635475 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:48.635478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:48.635619 (+ 141us) server_negotiation.cc:407] Connection header received
0812 18:59:48.635697 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:48.635699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:48.635729 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:48.635785 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 18:59:48.636348 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.636761 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.637417 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.637581 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.637644 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.638759 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 18:59:48.638761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 18:59:48.638763 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 18:59:48.638780 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 18:59:48.639993 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:48.640383 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:48.640385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:48.640386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:48.640449 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 18:59:48.640664 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 18:59:48.640666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 18:59:48.640667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 18:59:48.640834 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 18:59:48.640924 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:48.641093 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 18:59:48.641169 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 18:59:48.643725 9829 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 18:59:48.646898 9829 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 18:59:48.659991 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.654079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:33477 (local address 127.0.0.1:51918)
0812 18:59:48.654941 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:48.654987 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:48.655149 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:48.655799 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:48.655803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:48.655833 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:48.655996 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.656003 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.657413 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.657416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.658584 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.658593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.659387 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.659391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.659593 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.659625 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 18:59:48.659737 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 18:59:48.659849 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":52,"spinlock_wait_cycles":4736,"thread_start_us":69,"threads_started":1}
I20260812 18:59:48.660010 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.654185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.29.112.193:33477)
0812 18:59:48.654717 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:48.654729 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:48.655070 (+ 341us) server_negotiation.cc:407] Connection header received
0812 18:59:48.655322 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:48.655327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:48.655511 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:48.655635 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:48.656134 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.657269 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.658693 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.659258 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.659400 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.659536 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:48.659770 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 18:59:48.659863 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":138,"threads_started":1}
I20260812 18:59:48.660545 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.655206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.29.112.194:32895)
0812 18:59:48.655607 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:48.655618 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:48.656306 (+ 688us) server_negotiation.cc:407] Connection header received
0812 18:59:48.656419 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:48.656424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:48.656569 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:48.656715 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:48.656892 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.657733 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.658813 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.659616 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.659725 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.659831 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:48.660307 (+ 476us) server_negotiation.cc:299] Negotiation successful
0812 18:59:48.660391 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":133,"threads_started":1}
I20260812 18:59:48.660866 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.654897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:32895 (local address 127.0.0.1:42290)
0812 18:59:48.656202 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:48.656212 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:48.656313 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:48.656693 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:48.656696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:48.656712 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:48.656797 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.656803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.657892 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.657895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.658696 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.658707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.659893 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.659899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.659993 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.660016 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 18:59:48.660672 (+ 656us) client_negotiation.cc:240] Negotiation successful
0812 18:59:48.660744 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1258,"thread_start_us":94,"threads_started":1}
I20260812 18:59:48.662873 9979 tablet_service.cc:1467] Processing CreateTablet for tablet 58853e39c033462d9289a159458986ac (DEFAULT_TABLE table=test-table [id=c03f55d52953453f85be79de02a03d45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:48.663228 10130 tablet_service.cc:1467] Processing CreateTablet for tablet 58853e39c033462d9289a159458986ac (DEFAULT_TABLE table=test-table [id=c03f55d52953453f85be79de02a03d45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:48.663821 9979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58853e39c033462d9289a159458986ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:48.664179 10130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58853e39c033462d9289a159458986ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:48.670585 10385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.654275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37814 (local address 127.29.112.195:36189)
0812 18:59:48.654679 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:48.654686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:48.655217 (+ 531us) server_negotiation.cc:407] Connection header received
0812 18:59:48.655389 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:48.655393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:48.655530 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:48.655641 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:48.656134 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.657272 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.659937 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.660723 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.660828 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.660935 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:48.670336 (+ 9401us) server_negotiation.cc:299] Negotiation successful
0812 18:59:48.670433 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":75,"threads_started":1}
I20260812 18:59:48.670598 10388 tablet_bootstrap.cc:492] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965: Bootstrap starting.
I20260812 18:59:48.670800 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.654182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:36189 (local address 127.0.0.1:37814)
0812 18:59:48.654941 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:48.654987 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:48.655258 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:48.655797 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:48.655802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:48.655835 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:48.656001 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.656005 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.658679 (+ 2674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.658680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.659805 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.659816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.661007 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.661010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.669943 (+ 8933us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.669978 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 18:59:48.670061 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 18:59:48.670132 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":93,"spinlock_wait_cycles":6016,"thread_start_us":84,"threads_started":1}
I20260812 18:59:48.673301 10388 tablet_bootstrap.cc:654] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:48.673693 10390 tablet_bootstrap.cc:492] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06: Bootstrap starting.
I20260812 18:59:48.673429 10288 tablet_service.cc:1467] Processing CreateTablet for tablet 58853e39c033462d9289a159458986ac (DEFAULT_TABLE table=test-table [id=c03f55d52953453f85be79de02a03d45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 18:59:48.674176 10388 log.cc:824] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:48.674477 10288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58853e39c033462d9289a159458986ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 18:59:48.676131 10390 tablet_bootstrap.cc:654] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:48.676218 10388 tablet_bootstrap.cc:492] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965: No bootstrap required, opened a new log
I20260812 18:59:48.676395 10388 ts_tablet_manager.cc:1397] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 18:59:48.676842 10390 log.cc:824] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:48.678534 10390 tablet_bootstrap.cc:492] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06: No bootstrap required, opened a new log
I20260812 18:59:48.678730 10390 ts_tablet_manager.cc:1397] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 18:59:48.680258 10388 raft_consensus.cc:348] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a594eaacedd4f61a84df1e171fb3965" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 33477 } } peers { permanent_uuid: "aaadb68ee8e04ce19210893d90afdcc9" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36189 } } peers { permanent_uuid: "f4e1e69e49ec4cef98d24be00926bf06" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32895 } }
I20260812 18:59:48.680675 10388 raft_consensus.cc:374] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:48.680737 10388 raft_consensus.cc:729] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a594eaacedd4f61a84df1e171fb3965, State: Initialized, Role: FOLLOWER
I20260812 18:59:48.681197 10394 tablet_bootstrap.cc:492] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9: Bootstrap starting.
I20260812 18:59:48.681219 10388 consensus_queue.cc:260] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a594eaacedd4f61a84df1e171fb3965" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 33477 } } peers { permanent_uuid: "aaadb68ee8e04ce19210893d90afdcc9" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36189 } } peers { permanent_uuid: "f4e1e69e49ec4cef98d24be00926bf06" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32895 } }
I20260812 18:59:48.682122 10388 ts_tablet_manager.cc:1428] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 18:59:48.682478 10052 heartbeater.cc:503] Master 127.29.112.254:33607 was elected leader, sending a full tablet report...
I20260812 18:59:48.682340 10390 raft_consensus.cc:348] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a594eaacedd4f61a84df1e171fb3965" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 33477 } } peers { permanent_uuid: "aaadb68ee8e04ce19210893d90afdcc9" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36189 } } peers { permanent_uuid: "f4e1e69e49ec4cef98d24be00926bf06" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32895 } }
I20260812 18:59:48.682571 10390 raft_consensus.cc:374] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:48.682624 10390 raft_consensus.cc:729] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e1e69e49ec4cef98d24be00926bf06, State: Initialized, Role: FOLLOWER
I20260812 18:59:48.683037 10390 consensus_queue.cc:260] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a594eaacedd4f61a84df1e171fb3965" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 33477 } } peers { permanent_uuid: "aaadb68ee8e04ce19210893d90afdcc9" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36189 } } peers { permanent_uuid: "f4e1e69e49ec4cef98d24be00926bf06" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32895 } }
I20260812 18:59:48.683938 10390 ts_tablet_manager.cc:1428] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 18:59:48.684118 10394 tablet_bootstrap.cc:654] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9: Neither blocks nor log segments found. Creating new log.
I20260812 18:59:48.684288 10206 heartbeater.cc:503] Master 127.29.112.254:33607 was elected leader, sending a full tablet report...
I20260812 18:59:48.685196 10394 log.cc:824] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9: Log is configured to *not* fsync() on all Append() calls
I20260812 18:59:48.687389 10394 tablet_bootstrap.cc:492] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9: No bootstrap required, opened a new log
I20260812 18:59:48.687530 10394 ts_tablet_manager.cc:1397] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 18:59:48.690565 10394 raft_consensus.cc:348] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a594eaacedd4f61a84df1e171fb3965" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 33477 } } peers { permanent_uuid: "aaadb68ee8e04ce19210893d90afdcc9" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36189 } } peers { permanent_uuid: "f4e1e69e49ec4cef98d24be00926bf06" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32895 } }
I20260812 18:59:48.690830 10394 raft_consensus.cc:374] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 18:59:48.690876 10394 raft_consensus.cc:729] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaadb68ee8e04ce19210893d90afdcc9, State: Initialized, Role: FOLLOWER
I20260812 18:59:48.691155 10394 consensus_queue.cc:260] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a594eaacedd4f61a84df1e171fb3965" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 33477 } } peers { permanent_uuid: "aaadb68ee8e04ce19210893d90afdcc9" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36189 } } peers { permanent_uuid: "f4e1e69e49ec4cef98d24be00926bf06" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32895 } }
I20260812 18:59:48.691668 10369 heartbeater.cc:503] Master 127.29.112.254:33607 was elected leader, sending a full tablet report...
I20260812 18:59:48.691946 10394 ts_tablet_manager.cc:1428] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 18:59:48.694444 10399 raft_consensus.cc:482] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 18:59:48.694603 10399 raft_consensus.cc:504] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a594eaacedd4f61a84df1e171fb3965" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 33477 } } peers { permanent_uuid: "aaadb68ee8e04ce19210893d90afdcc9" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36189 } } peers { permanent_uuid: "f4e1e69e49ec4cef98d24be00926bf06" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32895 } }
I20260812 18:59:48.695498 10399 leader_election.cc:288] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a594eaacedd4f61a84df1e171fb3965 (127.29.112.193:33477), f4e1e69e49ec4cef98d24be00926bf06 (127.29.112.194:32895)
I20260812 18:59:48.699271 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.695692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:32895 (local address 127.29.112.195:35993)
0812 18:59:48.695871 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:48.695885 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:48.695963 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:48.696280 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:48.696283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:48.696301 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:48.696428 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.696434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.697207 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.697210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.698217 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.698225 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.698858 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.698863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.698962 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.698985 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 18:59:48.699080 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 18:59:48.699133 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260812 18:59:48.699298 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.695796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:35993 (local address 127.29.112.194:32895)
0812 18:59:48.696037 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:48.696042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:48.696058 (+ 16us) server_negotiation.cc:407] Connection header received
0812 18:59:48.696105 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:48.696110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:48.696165 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:48.696249 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:48.696549 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.697105 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.698326 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.698745 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.698835 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.698907 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:48.699103 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 18:59:48.699187 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 18:59:48.699397 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.696039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:40661 (local address 127.29.112.193:33477)
0812 18:59:48.696140 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:48.696144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:48.696164 (+ 20us) server_negotiation.cc:407] Connection header received
0812 18:59:48.696294 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:48.696297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:48.696337 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:48.696416 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 18:59:48.696691 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.697255 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.698427 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.698783 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.698912 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.698973 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:48.699211 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 18:59:48.699275 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 18:59:48.699431 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.695692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:33477 (local address 127.29.112.195:40661)
0812 18:59:48.696086 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:48.696100 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 18:59:48.696189 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 18:59:48.696436 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 18:59:48.696438 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 18:59:48.696452 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 18:59:48.696586 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.696592 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.697362 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.697364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.698312 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 18:59:48.698323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.698925 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.698929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 18:59:48.699079 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.699103 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 18:59:48.699180 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 18:59:48.699241 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":31,"thread_start_us":84,"threads_started":1}
I20260812 18:59:48.699860 10155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58853e39c033462d9289a159458986ac" candidate_uuid: "aaadb68ee8e04ce19210893d90afdcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e1e69e49ec4cef98d24be00926bf06" is_pre_election: true
I20260812 18:59:48.699959 10000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58853e39c033462d9289a159458986ac" candidate_uuid: "aaadb68ee8e04ce19210893d90afdcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a594eaacedd4f61a84df1e171fb3965" is_pre_election: true
I20260812 18:59:48.700103 10155 raft_consensus.cc:2454] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaadb68ee8e04ce19210893d90afdcc9 in term 0.
I20260812 18:59:48.700279 10000 raft_consensus.cc:2454] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaadb68ee8e04ce19210893d90afdcc9 in term 0.
I20260812 18:59:48.700613 10234 leader_election.cc:302] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaadb68ee8e04ce19210893d90afdcc9, f4e1e69e49ec4cef98d24be00926bf06; no voters:
I20260812 18:59:48.700838 10399 raft_consensus.cc:2781] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 18:59:48.700909 10399 raft_consensus.cc:482] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 18:59:48.700950 10399 raft_consensus.cc:3037] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:48.701861 10399 raft_consensus.cc:504] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a594eaacedd4f61a84df1e171fb3965" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 33477 } } peers { permanent_uuid: "aaadb68ee8e04ce19210893d90afdcc9" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36189 } } peers { permanent_uuid: "f4e1e69e49ec4cef98d24be00926bf06" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32895 } }
I20260812 18:59:48.702163 10399 leader_election.cc:288] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [CANDIDATE]: Term 1 election: Requested vote from peers 3a594eaacedd4f61a84df1e171fb3965 (127.29.112.193:33477), f4e1e69e49ec4cef98d24be00926bf06 (127.29.112.194:32895)
I20260812 18:59:48.702457 10000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58853e39c033462d9289a159458986ac" candidate_uuid: "aaadb68ee8e04ce19210893d90afdcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a594eaacedd4f61a84df1e171fb3965"
I20260812 18:59:48.702513 10155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58853e39c033462d9289a159458986ac" candidate_uuid: "aaadb68ee8e04ce19210893d90afdcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e1e69e49ec4cef98d24be00926bf06"
I20260812 18:59:48.702592 10000 raft_consensus.cc:3037] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:48.702616 10155 raft_consensus.cc:3037] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06 [term 0 FOLLOWER]: Advancing to term 1
I20260812 18:59:48.703536 10000 raft_consensus.cc:2454] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaadb68ee8e04ce19210893d90afdcc9 in term 1.
I20260812 18:59:48.703627 10155 raft_consensus.cc:2454] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaadb68ee8e04ce19210893d90afdcc9 in term 1.
I20260812 18:59:48.703840 10236 leader_election.cc:302] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a594eaacedd4f61a84df1e171fb3965, aaadb68ee8e04ce19210893d90afdcc9; no voters:
I20260812 18:59:48.704044 10399 raft_consensus.cc:2781] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 18:59:48.704252 10399 raft_consensus.cc:686] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 1 LEADER]: Becoming Leader. State: Replica: aaadb68ee8e04ce19210893d90afdcc9, State: Running, Role: LEADER
I20260812 18:59:48.704532 10399 consensus_queue.cc:237] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a594eaacedd4f61a84df1e171fb3965" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 33477 } } peers { permanent_uuid: "aaadb68ee8e04ce19210893d90afdcc9" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36189 } } peers { permanent_uuid: "f4e1e69e49ec4cef98d24be00926bf06" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32895 } }
I20260812 18:59:48.707137 9829 catalog_manager.cc:5495] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to aaadb68ee8e04ce19210893d90afdcc9 (127.29.112.195). New cstate: current_term: 1 leader_uuid: "aaadb68ee8e04ce19210893d90afdcc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a594eaacedd4f61a84df1e171fb3965" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 33477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaadb68ee8e04ce19210893d90afdcc9" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4e1e69e49ec4cef98d24be00926bf06" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32895 } health_report { overall_health: UNKNOWN } } }
W20260812 18:59:48.711321 10055 tablet.cc:2369] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 18:59:48.720348 10385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.717139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37820 (local address 127.29.112.195:36189)
0812 18:59:48.717259 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 18:59:48.717264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 18:59:48.717313 (+ 49us) server_negotiation.cc:407] Connection header received
0812 18:59:48.717458 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 18:59:48.717460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 18:59:48.717510 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 18:59:48.717604 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 18:59:48.717900 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.718324 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.719176 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 18:59:48.719376 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 18:59:48.719452 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 18:59:48.719706 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 18:59:48.720049 (+ 343us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 18:59:48.720134 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 18:59:48.720205 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 18:59:48.720257 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 18:59:48.724838 10000 raft_consensus.cc:1261] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965 [term 1 FOLLOWER]: Refusing update from remote peer aaadb68ee8e04ce19210893d90afdcc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:48.724844 10155 raft_consensus.cc:1261] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06 [term 1 FOLLOWER]: Refusing update from remote peer aaadb68ee8e04ce19210893d90afdcc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 18:59:48.725435 10399 consensus_queue.cc:1035] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a594eaacedd4f61a84df1e171fb3965" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 33477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:48.725636 10403 consensus_queue.cc:1035] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e1e69e49ec4cef98d24be00926bf06" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 32895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 18:59:48.734395 10408 mvcc.cc:205] Tried to move back new op lower bound from 7317754629011800064 to 7317754628936093696. Current Snapshot: MvccSnapshot[applied={T|T < 7317754629011800064}]
I20260812 18:59:48.735266 10407 mvcc.cc:205] Tried to move back new op lower bound from 7317754629011800064 to 7317754628936093696. Current Snapshot: MvccSnapshot[applied={T|T < 7317754629011800064 or (T in {7317754629011800064})}]
I20260812 18:59:48.739256 9829 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:48.739401 9829 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 18:59:48.741485 9829 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b96f82535a84a199897f824917334cf: Sending DeleteTablet for 3 replicas of tablet 58853e39c033462d9289a159458986ac
I20260812 18:59:48.742118 9979 tablet_service.cc:1514] Processing DeleteTablet for tablet 58853e39c033462d9289a159458986ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:48 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:51918
I20260812 18:59:48.742203 10288 tablet_service.cc:1514] Processing DeleteTablet for tablet 58853e39c033462d9289a159458986ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:48 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:37814
I20260812 18:59:48.742302 10130 tablet_service.cc:1514] Processing DeleteTablet for tablet 58853e39c033462d9289a159458986ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 18:59:48 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:42290
I20260812 18:59:48.742713 10415 tablet_replica.cc:330] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965: stopping tablet replica
I20260812 18:59:48.742864 10420 tablet_replica.cc:330] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06: stopping tablet replica
I20260812 18:59:48.742990 10415 raft_consensus.cc:2229] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:48.743225 10420 raft_consensus.cc:2229] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 18:59:48.743251 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 9905
I20260812 18:59:48.743257 10415 raft_consensus.cc:2258] T 58853e39c033462d9289a159458986ac P 3a594eaacedd4f61a84df1e171fb3965 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:48.743551 10417 tablet_replica.cc:330] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9: stopping tablet replica
I20260812 18:59:48.743763 10417 raft_consensus.cc:2229] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 18:59:48.744050 10417 raft_consensus.cc:2258] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:48.746017 10420 raft_consensus.cc:2258] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 18:59:48.747401 10420 ts_tablet_manager.cc:1905] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:48.749415 10420 ts_tablet_manager.cc:1918] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:48.749531 10420 log.cc:1196] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/58853e39c033462d9289a159458986ac
I20260812 18:59:48.749912 10420 ts_tablet_manager.cc:1939] T 58853e39c033462d9289a159458986ac P f4e1e69e49ec4cef98d24be00926bf06: Deleting consensus metadata
I20260812 18:59:48.750720 10417 ts_tablet_manager.cc:1905] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 18:59:48.750978 9810 catalog_manager.cc:4841] TS f4e1e69e49ec4cef98d24be00926bf06 (127.29.112.194:32895): tablet 58853e39c033462d9289a159458986ac (table test-table [id=c03f55d52953453f85be79de02a03d45]) successfully deleted
I20260812 18:59:48.752843 10417 ts_tablet_manager.cc:1918] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 18:59:48.752954 10417 log.cc:1196] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/58853e39c033462d9289a159458986ac
I20260812 18:59:48.753261 10417 ts_tablet_manager.cc:1939] T 58853e39c033462d9289a159458986ac P aaadb68ee8e04ce19210893d90afdcc9: Deleting consensus metadata
W20260812 18:59:48.753723 9813 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:33477 (error 108)
I20260812 18:59:48.754189 9811 catalog_manager.cc:4841] TS aaadb68ee8e04ce19210893d90afdcc9 (127.29.112.195:36189): tablet 58853e39c033462d9289a159458986ac (table test-table [id=c03f55d52953453f85be79de02a03d45]) successfully deleted
I20260812 18:59:48.754570 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 18:59:48.754320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:33477 (local address 127.0.0.1:51920)
0812 18:59:48.754439 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 18:59:48.754500 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 18:59:48.754710 9813 catalog_manager.cc:4584] TS 3a594eaacedd4f61a84df1e171fb3965 (127.29.112.193:33477): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 58853e39c033462d9289a159458986ac: Network error: Client connection negotiation failed: client connection to 127.29.112.193:33477: connect: Connection refused (error 111)
I20260812 18:59:48.754808 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 10060
I20260812 18:59:48.775872 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 10214
I20260812 18:59:48.782814 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 9793
2026-08-12T18:59:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2208 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 18:59:48 dist-test-slave-6nmv krb5kdc[10424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 18:59:48 dist-test-slave-6nmv krb5kdc[10424](info): set up 2 sockets
Aug 12 18:59:48 dist-test-slave-6nmv krb5kdc[10424](info): commencing operation
krb5kdc: starting...
W20260812 18:59:49.854631 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 18:59:49 dist-test-slave-6nmv krb5kdc[10424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T18:59:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T18:59:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.112.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.112.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/rangeradmin_127.29.112.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/rangeradmin_127.29.112.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.112.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.112.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/rangerlookup_127.29.112.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/rangerlookup_127.29.112.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.112.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.112.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/HTTP_127.29.112.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/HTTP_127.29.112.212@KRBTEST.COM.keytab.
I20260812 18:59:50.016816 30147 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmcQeqc/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/postgres -l logfile start
2026-08-12 18:59:52.375 UTC [10459] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 18:59:52.375 UTC [10459] LOG: listening on IPv4 address "127.29.112.212", port 57949
2026-08-12 18:59:52.379 UTC [10459] LOG: listening on Unix socket "/tmp/.s.PGSQL.57949"
2026-08-12 18:59:52.400 UTC [10462] LOG: database system was shut down at 2026-08-12 18:59:51 UTC
2026-08-12 18:59:52.404 UTC [10459] LOG: database system is ready to accept connections
I20260812 18:59:53.463923 30147 mini_postgres.cc:96] Postgres bound to 57949
2026-08-12 18:59:53.472 UTC [10470] FATAL: database "slave" does not exist
127.29.112.212:57949 - accepting connections
I20260812 18:59:53.473337 30147 mini_ranger.cc:161] Starting Ranger...
I20260812 18:59:53.490267 30147 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 18:59:53.921307 30147 mini_ranger.cc:87] Created ranger Postgres database
I20260812 18:59:53.921407 30147 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-admin
2026-08-12 18:59:54,218 [I] DB FLAVOR :POSTGRES
2026-08-12 18:59:54,218 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 18:59:54,218 [I] Checking connection..
2026-08-12 18:59:54,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 18:59:54,457 [I] Checking connection passed.
2026-08-12 18:59:54,457 [I] --------- Verifying version history table ---------
2026-08-12 18:59:54,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 18:59:54,682 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 18:59:54,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 18:59:54,925 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 18:59:54,925 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 18:59:54,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 18:59:55.163 UTC [10547] WARNING: there is no transaction in progress
2026-08-12 18:59:55,170 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 18:59:55,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 18:59:55,410 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 18:59:55,410 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 18:59:55,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 18:59:55,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 18:59:55,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6nmv.c.gcp-upstream.internal','N') ;"
2026-08-12 18:59:56,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 18:59:56,350 [I] Table x_portal_user does not exist in database ranger
2026-08-12 18:59:56,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 18:59:56,561 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 18:59:56,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 18:59:56,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 18:59:57,040 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 18:59:57,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 18:59:57.945 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:57.960 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:57.974 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:57.988 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.002 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.018 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.067 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.073 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.086 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.095 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.104 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.117 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.125 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.131 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.137 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58.143 UTC [10677] WARNING: there is no transaction in progress
2026-08-12 18:59:58,754 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 18:59:58,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 18:59:59,002 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 18:59:59,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 18:59:59,256 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 18:59:59,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 18:59:59,482 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 18:59:59,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 18:59:59,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 18:59:59,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6nmv.c.gcp-upstream.internal';"
2026-08-12 19:00:00,207 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 19:00:00,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 19:00:00,446 [I] DB_PATCHES have already been applied
I20260812 19:00:00.451881 30147 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmcQeqc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 19:00:00.451951 30147 mini_ranger.cc:193] Using host: 127.29.112.212
I20260812 19:00:00.456269 30147 mini_ranger.cc:227] Ranger admin URL: http://127.29.112.212:51083
Aug 12, 2026 7:00:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:00:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:00:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 7:00:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 7:00:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 7:00:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 7:00:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 19:00:01.628212 30147 mini_ranger.cc:160] Time spent starting Ranger: real 8.155s user 0.006s sys 0.006s
Aug 12, 2026 7:00:22 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 7:00:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 12, 2026 7:00:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 7:00:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 7:00:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 19:00:24.240298 30147 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.112.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.112.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/rangerkms_127.29.112.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/rangerkms_127.29.112.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.112.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.112.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.112.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/HTTP_127.29.112.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.112.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/HTTP_127.29.112.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 19:00:24 dist-test-slave-6nmv krb5kdc[10424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561224, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 19:00:24.324509 30147 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 19:00:24.334306 30147 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 19:00:25.266945 30147 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 19:00:25.267083 30147 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-kms
I20260812 19:00:25.267230 30147 mini_ranger_kms.cc:227] Using postgres at 127.29.112.212:57949
2026-08-12 19:00:26,148 [I] DB FLAVOR :POSTGRES
2026-08-12 19:00:26,149 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 19:00:26,149 [I] Checking connection
2026-08-12 19:00:26,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 19:00:26,476 [I] connection success
2026-08-12 19:00:26,476 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 19:00:26,476 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 19:00:26,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 19:00:26,805 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 19:00:26,806 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 19:00:26,806 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 19:00:26,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:57949/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 19:00:27,090 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 19:00:27.584235 30147 mini_ranger_kms.cc:310] Created kms service
I20260812 19:00:27.816768 30147 mini_ranger_kms.cc:326] Created kudu user
I20260812 19:00:27.865978 30147 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 19:00:27 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561224, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:00:28.292395 30147 mini_ranger_kms.cc:384] Added ranger policy
I20260812 19:00:28.292572 30147 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 19:00:28.292605 30147 mini_ranger_kms.cc:240] Using host: 127.29.112.212
I20260812 19:00:28.297044 30147 mini_ranger_kms.cc:276] Ranger KMS PID: 10891
I20260812 19:00:28.297200 30147 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.112.212:54921
Aug 12, 2026 7:00:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:00:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:00:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 7:00:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 7:00:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 7:00:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 19:00:30.660862 30147 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.336s user 0.007s sys 0.681s
I20260812 19:00:30.661027 30147 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 19:00:30.661166 30147 mini_ranger_kms.cc:401] 127.29.112.212:54921/kms/v1/keys
Aug 12 19:00:30 dist-test-slave-6nmv krb5kdc[10424](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786561230, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 19:00:31 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561230, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.212@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
Aug 12 19:00:31 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561230, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.212@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
Aug 12 19:00:33 dist-test-slave-6nmv krb5kdc[10424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:00:33.825127 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:39283
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:39283
--ranger_config_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:54921/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:00:33.959870 10953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:00:33.960126 10953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:00:33.960166 10953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:00:33.964224 10953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:00:33.964283 10953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:00:33.964303 10953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:00:33.964319 10953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:00:33.964334 10953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:00:33.969403 10953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.112.212:54921/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:39283
--ranger_config_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:39283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:00:33.970305 10953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:00:33.971205 10953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:00:33.980166 10959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:00:33.980317 10953 server_base.cc:1034] running on GCE node
W20260812 19:00:33.980247 10963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:00:33.980173 10960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:00:33.981043 10953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:00:33.982173 10953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:00:33.983355 10953 hybrid_clock.cc:648] HybridClock initialized: now 1786561233983312 us; error 28 us; skew 500 ppm
Aug 12 19:00:33 dist-test-slave-6nmv krb5kdc[10424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561233, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:00:33.986373 10953 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:00:33.987291 10953 webserver.cc:460] Webserver started at http://127.29.112.254:34171/ using document root <none> and password file <none>
I20260812 19:00:33.987826 10953 fs_manager.cc:359] Metadata directory not provided
I20260812 19:00:33.987906 10953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:00:33.988130 10953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:00:33 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561233, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:00:34.034549 10953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "7b41ce05fab247d79a98ab2a625c9b2a"
format_stamp: "Formatted at 2026-08-12 19:00:34 on dist-test-slave-6nmv"
server_key: "e379fd6340f8c9d242efbb75831b2ec9"
server_key_iv: "d85a88bdaa81eabd070006061a98b227"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:34.035295 10953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "7b41ce05fab247d79a98ab2a625c9b2a"
format_stamp: "Formatted at 2026-08-12 19:00:34 on dist-test-slave-6nmv"
server_key: "e379fd6340f8c9d242efbb75831b2ec9"
server_key_iv: "d85a88bdaa81eabd070006061a98b227"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:34.039317 10953 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:00:34.057103 10972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:00:34.058588 10953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:00:34.058730 10953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "7b41ce05fab247d79a98ab2a625c9b2a"
format_stamp: "Formatted at 2026-08-12 19:00:34 on dist-test-slave-6nmv"
server_key: "e379fd6340f8c9d242efbb75831b2ec9"
server_key_iv: "d85a88bdaa81eabd070006061a98b227"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:34.058871 10953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:00:34.069887 10953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:00:34.072923 10953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:00:34.073132 10953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:00:34.083114 10953 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:39283
I20260812 19:00:34.083133 11031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:39283 every 8 connection(s)
I20260812 19:00:34.084151 10953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:00:34.087471 11032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:00:34.093251 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 10953
I20260812 19:00:34.093405 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:00:34.093708 30147 external_mini_cluster.cc:1428] Setting key c953d7496ad2e3f868c5915fa93104e3
I20260812 19:00:34.094784 11032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a: Bootstrap starting.
I20260812 19:00:34.097653 11032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a: Neither blocks nor log segments found. Creating new log.
I20260812 19:00:34.098665 11032 log.cc:824] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a: Log is configured to *not* fsync() on all Append() calls
I20260812 19:00:34.100706 11032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a: No bootstrap required, opened a new log
Aug 12 19:00:34 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:00:34.105314 11032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b41ce05fab247d79a98ab2a625c9b2a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39283 } }
I20260812 19:00:34.105599 11032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:00:34.105687 11032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b41ce05fab247d79a98ab2a625c9b2a, State: Initialized, Role: FOLLOWER
I20260812 19:00:34.106257 11032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b41ce05fab247d79a98ab2a625c9b2a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39283 } }
I20260812 19:00:34.106407 11032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:00:34.106496 11032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:00:34.106594 11032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:00:34.107553 11032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b41ce05fab247d79a98ab2a625c9b2a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39283 } }
I20260812 19:00:34.107879 11032 leader_election.cc:302] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b41ce05fab247d79a98ab2a625c9b2a; no voters:
I20260812 19:00:34.108197 11032 leader_election.cc:288] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:00:34.108351 11039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:00:34.108646 11039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [term 1 LEADER]: Becoming Leader. State: Replica: 7b41ce05fab247d79a98ab2a625c9b2a, State: Running, Role: LEADER
I20260812 19:00:34.108981 11039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b41ce05fab247d79a98ab2a625c9b2a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39283 } }
I20260812 19:00:34.109514 11032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:00:34.110116 11039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b41ce05fab247d79a98ab2a625c9b2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b41ce05fab247d79a98ab2a625c9b2a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39283 } } }
I20260812 19:00:34.110311 11039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [sys.catalog]: This master's current role is: LEADER
I20260812 19:00:34.110442 11036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:34.095140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35856 (local address 127.29.112.254:39283)
0812 19:00:34.095607 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 19:00:34.095621 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:00:34.095654 (+ 33us) server_negotiation.cc:407] Connection header received
0812 19:00:34.096374 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:00:34.096391 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:00:34.096711 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:00:34.097028 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:00:34.097774 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:34.099150 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:34.100284 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:34.101038 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:34.101213 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:34.104453 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:00:34.104462 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:00:34.104478 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:00:34.104518 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:00:34.106804 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:00:34.107327 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:00:34.107332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:00:34.107337 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:00:34.107468 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:00:34.107784 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:00:34.107787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:00:34.107789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:00:34.108303 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:00:34.109156 (+ 853us) server_negotiation.cc:1035] Waiting for connection context
0812 19:00:34.109292 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 19:00:34.109550 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":122,"threads_started":1}
I20260812 19:00:34.111784 11039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b41ce05fab247d79a98ab2a625c9b2a. Latest consensus state: current_term: 1 leader_uuid: "7b41ce05fab247d79a98ab2a625c9b2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b41ce05fab247d79a98ab2a625c9b2a" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 39283 } } }
I20260812 19:00:34.111939 11039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a [sys.catalog]: This master's current role is: LEADER
I20260812 19:00:34.112053 11032 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 19:00:34.113088 11049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:00:34.116279 11049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:00:34.125172 11049 catalog_manager.cc:1347] Generated new cluster ID: 1867626685c840dbb5254d795c4701ad
I20260812 19:00:34.125252 11049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:00:34.142093 11049 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:00:34.142946 11049 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:00:34.156347 11049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a: Generated new TSK 0
I20260812 19:00:34.157060 11049 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 19:00:34.897295 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:54921/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:39283
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:00:35.045168 11089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:00:35.045457 11089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:00:35.045513 11089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:00:35.049320 11089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:00:35.049371 11089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:00:35.049422 11089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:00:35.056361 11089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.112.212:54921/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:39283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:00:35.057559 11089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:00:35.058696 11089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:00:35.070596 11100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:00:35.070942 11089 server_base.cc:1034] running on GCE node
W20260812 19:00:35.071069 11101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:00:35.070685 11103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:00:35.071643 11089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:00:35.072378 11089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:00:35.073551 11089 hybrid_clock.cc:648] HybridClock initialized: now 1786561235073514 us; error 54 us; skew 500 ppm
Aug 12 19:00:35 dist-test-slave-6nmv krb5kdc[10424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:00:35.077114 11089 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:00:35.078243 11089 webserver.cc:460] Webserver started at http://127.29.112.193:33591/ using document root <none> and password file <none>
I20260812 19:00:35.078918 11089 fs_manager.cc:359] Metadata directory not provided
I20260812 19:00:35.079005 11089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:00:35.079270 11089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:00:35 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:00:35.095139 11089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "e13762bf55f141feab0e244aa6d35d5a"
format_stamp: "Formatted at 2026-08-12 19:00:35 on dist-test-slave-6nmv"
server_key: "c2e3aa1b834b4d676ee22f48d9c1b53e"
server_key_iv: "5db7c56c391693b31ef0e755d16889bb"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:35.095830 11089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "e13762bf55f141feab0e244aa6d35d5a"
format_stamp: "Formatted at 2026-08-12 19:00:35 on dist-test-slave-6nmv"
server_key: "c2e3aa1b834b4d676ee22f48d9c1b53e"
server_key_iv: "5db7c56c391693b31ef0e755d16889bb"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:35.100292 11089 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:00:35.115830 11111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:00:35.117461 11089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 19:00:35.117578 11089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "e13762bf55f141feab0e244aa6d35d5a"
format_stamp: "Formatted at 2026-08-12 19:00:35 on dist-test-slave-6nmv"
server_key: "c2e3aa1b834b4d676ee22f48d9c1b53e"
server_key_iv: "5db7c56c391693b31ef0e755d16889bb"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:35.117692 11089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:00:35.144353 11089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:00:35.147948 11089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:00:35.148175 11089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:00:35.148885 11089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:00:35.149942 11089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:00:35.150074 11089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:00:35.150198 11089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:00:35.150283 11089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:00:35.165112 11089 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:46673
I20260812 19:00:35.165272 11247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:46673 every 8 connection(s)
I20260812 19:00:35.166095 11089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:00:35.175235 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 11089
I20260812 19:00:35.175388 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 19:00:35.175736 30147 external_mini_cluster.cc:1428] Setting key e8c98031a961674d44c80562f3eb9f14
Aug 12 19:00:35 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:00:35.185312 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.169699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:53181 (local address 127.29.112.254:39283)
0812 19:00:35.171732 (+ 2033us) server_negotiation.cc:206] Beginning negotiation
0812 19:00:35.171737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:00:35.171753 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:00:35.171799 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:00:35.171802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:00:35.171851 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:00:35.171949 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:00:35.173516 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.174420 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.176400 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.177414 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.177494 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.180680 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:00:35.180683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:00:35.180686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:00:35.180711 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:00:35.182819 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:00:35.183487 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:00:35.183491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:00:35.183493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:00:35.183583 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:00:35.184016 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:00:35.184020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:00:35.184022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:00:35.184237 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:00:35.184315 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 19:00:35.185020 (+ 705us) server_negotiation.cc:299] Negotiation successful
0812 19:00:35.185153 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1930,"thread_start_us":133,"threads_started":1}
I20260812 19:00:35.186996 11253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.168617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:39283 (local address 127.29.112.193:53181)
0812 19:00:35.170180 (+ 1563us) negotiation.cc:107] Waiting for socket to connect
0812 19:00:35.170232 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 19:00:35.171302 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:00:35.172273 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:00:35.172282 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:00:35.172679 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:00:35.173332 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.173352 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.174577 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.174582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.176253 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.176265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.177693 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.177698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.177946 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.178619 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:00:35.178644 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:00:35.180500 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:00:35.182991 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:00:35.182996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:00:35.183009 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:00:35.183347 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:00:35.183699 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:00:35.183701 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:00:35.183703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:00:35.183876 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:00:35.184405 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:00:35.184410 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:00:35.184704 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 19:00:35.185732 (+ 1028us) client_negotiation.cc:240] Negotiation successful
0812 19:00:35.186034 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1340,"thread_start_us":154,"threads_started":1}
I20260812 19:00:35.188567 11249 heartbeater.cc:347] Connected to a master server at 127.29.112.254:39283
I20260812 19:00:35.188777 11249 heartbeater.cc:464] Registering TS with master...
I20260812 19:00:35.189211 11249 heartbeater.cc:511] Master 127.29.112.254:39283 requested a full tablet report, sending...
I20260812 19:00:35.190564 10991 ts_manager.cc:194] Registered new tserver with Master: e13762bf55f141feab0e244aa6d35d5a (127.29.112.193:46673)
I20260812 19:00:35.191773 10991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:53181
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 19:00:35.216857 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:54921/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:39283
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:00:35 dist-test-slave-6nmv krb5kdc[10424](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 19:00:35.342227 11257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:00:35.342501 11257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:00:35.342598 11257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:00:35.346487 11257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:00:35.346566 11257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:00:35.346670 11257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:00:35.351279 11257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.112.212:54921/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:39283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:00:35.352190 11257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:00:35.353144 11257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:00:35.363894 11270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:00:35.364104 11274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:00:35.364329 11268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:00:35.364564 11257 server_base.cc:1034] running on GCE node
I20260812 19:00:35.364943 11257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:00:35.365523 11257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:00:35.366688 11257 hybrid_clock.cc:648] HybridClock initialized: now 1786561235366676 us; error 35 us; skew 500 ppm
Aug 12 19:00:35 dist-test-slave-6nmv krb5kdc[10424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:00:35.369551 11257 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:00:35.370622 11257 webserver.cc:460] Webserver started at http://127.29.112.194:35295/ using document root <none> and password file <none>
I20260812 19:00:35.371317 11257 fs_manager.cc:359] Metadata directory not provided
I20260812 19:00:35.371435 11257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:00:35.371701 11257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:00:35 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:00:35.385965 11257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "57faad74ee7e43358eed6ea875e3c3a8"
format_stamp: "Formatted at 2026-08-12 19:00:35 on dist-test-slave-6nmv"
server_key: "7d42346127e031933bc8faf505932823"
server_key_iv: "bcaf56bb09388efae33b76c811b75a63"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:35.386683 11257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "57faad74ee7e43358eed6ea875e3c3a8"
format_stamp: "Formatted at 2026-08-12 19:00:35 on dist-test-slave-6nmv"
server_key: "7d42346127e031933bc8faf505932823"
server_key_iv: "bcaf56bb09388efae33b76c811b75a63"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:35.390478 11257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:00:35.410321 11282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:00:35.412621 11257 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.001s
I20260812 19:00:35.412758 11257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "57faad74ee7e43358eed6ea875e3c3a8"
format_stamp: "Formatted at 2026-08-12 19:00:35 on dist-test-slave-6nmv"
server_key: "7d42346127e031933bc8faf505932823"
server_key_iv: "bcaf56bb09388efae33b76c811b75a63"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:35.412940 11257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:00:35.451162 11257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:00:35.459728 11257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:00:35.459959 11257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:00:35.460737 11257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:00:35.461700 11257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:00:35.461792 11257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:00:35.461951 11257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:00:35.462014 11257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:00:35.475983 11257 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:42225
I20260812 19:00:35.477135 11257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:00:35.479609 11421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:42225 every 8 connection(s)
I20260812 19:00:35.484757 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 11257
I20260812 19:00:35.484877 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 19:00:35.485193 30147 external_mini_cluster.cc:1428] Setting key 57681e4b0dca1bb911e2d0df2fb90209
Aug 12 19:00:35 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:00:35.498390 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.483195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:35147 (local address 127.29.112.254:39283)
0812 19:00:35.483328 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 19:00:35.483333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:00:35.484098 (+ 765us) server_negotiation.cc:407] Connection header received
0812 19:00:35.485865 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:00:35.485868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:00:35.485921 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:00:35.485985 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:00:35.488371 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.489200 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.490853 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.491184 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.491256 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.494536 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:00:35.494540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:00:35.494543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:00:35.494569 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:00:35.496233 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:00:35.496788 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:00:35.496791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:00:35.496793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:00:35.496874 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:00:35.497239 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:00:35.497242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:00:35.497244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:00:35.497433 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:00:35.497509 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 19:00:35.498119 (+ 610us) server_negotiation.cc:299] Negotiation successful
0812 19:00:35.498240 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 19:00:35.500173 11424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.482097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:39283 (local address 127.29.112.194:35147)
0812 19:00:35.483918 (+ 1821us) negotiation.cc:107] Waiting for socket to connect
0812 19:00:35.483985 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 19:00:35.485542 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:00:35.487058 (+ 1516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:00:35.487066 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:00:35.487416 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:00:35.488192 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.488215 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.489346 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.489349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.490721 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.490732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.491440 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.491443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.491712 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.492508 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:00:35.492528 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:00:35.494349 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:00:35.496362 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:00:35.496367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:00:35.496380 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:00:35.496677 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:00:35.496986 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:00:35.496988 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:00:35.496990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:00:35.497141 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:00:35.497569 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:00:35.497574 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:00:35.497856 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 19:00:35.498758 (+ 902us) client_negotiation.cc:240] Negotiation successful
0812 19:00:35.499100 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1613,"thread_start_us":142,"threads_started":1}
I20260812 19:00:35.501565 11422 heartbeater.cc:347] Connected to a master server at 127.29.112.254:39283
I20260812 19:00:35.501771 11422 heartbeater.cc:464] Registering TS with master...
I20260812 19:00:35.502364 11422 heartbeater.cc:511] Master 127.29.112.254:39283 requested a full tablet report, sending...
I20260812 19:00:35.503461 10991 ts_manager.cc:194] Registered new tserver with Master: 57faad74ee7e43358eed6ea875e3c3a8 (127.29.112.194:42225)
I20260812 19:00:35.504235 10991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:35147
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 19:00:35.545483 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:54921/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:39283
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:00:35.707453 11427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:00:35.707844 11427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:00:35.707974 11427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:00:35.713716 11427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:00:35.713858 11427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:00:35.714012 11427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:00:35.720922 11427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.112.212:54921/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:39283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:00:35.722311 11427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:00:35.723657 11427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:00:35.740381 11435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:00:35.740917 11438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:00:35.741981 11432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:00:35.744428 11427 server_base.cc:1034] running on GCE node
I20260812 19:00:35.744907 11427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:00:35.745658 11427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:00:35.746830 11427 hybrid_clock.cc:648] HybridClock initialized: now 1786561235746765 us; error 80 us; skew 500 ppm
Aug 12 19:00:35 dist-test-slave-6nmv krb5kdc[10424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:00:35.750784 11427 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:00:35.751961 11427 webserver.cc:460] Webserver started at http://127.29.112.195:35365/ using document root <none> and password file <none>
I20260812 19:00:35.752684 11427 fs_manager.cc:359] Metadata directory not provided
I20260812 19:00:35.752836 11427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:00:35.753129 11427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:00:35 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:00:35.780776 11427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "7c706ffcb50a44deb89f85769e010e93"
format_stamp: "Formatted at 2026-08-12 19:00:35 on dist-test-slave-6nmv"
server_key: "fea81fb1f1c706c78315ac02b17e976a"
server_key_iv: "b36d68314a4eb3cbaa5eb0230b6929db"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:35.782375 11427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "7c706ffcb50a44deb89f85769e010e93"
format_stamp: "Formatted at 2026-08-12 19:00:35 on dist-test-slave-6nmv"
server_key: "fea81fb1f1c706c78315ac02b17e976a"
server_key_iv: "b36d68314a4eb3cbaa5eb0230b6929db"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:35.793994 11427 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.004s sys 0.000s
I20260812 19:00:35.816944 11445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:00:35.818496 11427 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.003s
I20260812 19:00:35.818662 11427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "7c706ffcb50a44deb89f85769e010e93"
format_stamp: "Formatted at 2026-08-12 19:00:35 on dist-test-slave-6nmv"
server_key: "fea81fb1f1c706c78315ac02b17e976a"
server_key_iv: "b36d68314a4eb3cbaa5eb0230b6929db"
server_key_version: "kuduclusterkey@0"
I20260812 19:00:35.818790 11427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 12 19:00:35 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:00:35.861138 11427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:00:35.865054 11427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:00:35.865286 11427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:00:35.866158 11427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:00:35.867635 11427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:00:35.867734 11427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:00:35.867822 11427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:00:35.867882 11427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:00:35.884956 11427 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:45617
I20260812 19:00:35.887902 11583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:45617 every 8 connection(s)
I20260812 19:00:35.888445 11427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:00:35.897486 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 11427
I20260812 19:00:35.897603 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 19:00:35.898011 30147 external_mini_cluster.cc:1428] Setting key d482359bdbed2ceda93f86289b54bd40
Aug 12 19:00:35 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:00:35.909560 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.892573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:45975 (local address 127.29.112.254:39283)
0812 19:00:35.892700 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 19:00:35.892705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:00:35.893189 (+ 484us) server_negotiation.cc:407] Connection header received
0812 19:00:35.894605 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:00:35.894609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:00:35.894671 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:00:35.894731 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:00:35.896395 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.897167 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.902044 (+ 4877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.902330 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.902422 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.905552 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:00:35.905556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:00:35.905558 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:00:35.905584 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:00:35.907262 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:00:35.907827 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:00:35.907830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:00:35.907832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:00:35.907912 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:00:35.908262 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:00:35.908266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:00:35.908267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:00:35.908519 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:00:35.908586 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 19:00:35.909290 (+ 704us) server_negotiation.cc:299] Negotiation successful
0812 19:00:35.909402 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:00:35.912760 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.891468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:39283 (local address 127.29.112.195:45975)
0812 19:00:35.893034 (+ 1566us) negotiation.cc:107] Waiting for socket to connect
0812 19:00:35.893086 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 19:00:35.894218 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:00:35.895142 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:00:35.895151 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:00:35.895593 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:00:35.896220 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.896238 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.900492 (+ 4254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.900495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.901902 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.901913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.902607 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.902611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.902794 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.903452 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:00:35.903475 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:00:35.905330 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:00:35.907390 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:00:35.907395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:00:35.907420 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:00:35.907718 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:00:35.908010 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:00:35.908013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:00:35.908015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:00:35.908164 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:00:35.908682 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:00:35.908688 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:00:35.908950 (+ 262us) client_negotiation.cc:769] Sending connection context
0812 19:00:35.909970 (+ 1020us) client_negotiation.cc:240] Negotiation successful
0812 19:00:35.910351 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1353,"thread_start_us":228,"threads_started":1}
I20260812 19:00:35.914187 11586 heartbeater.cc:347] Connected to a master server at 127.29.112.254:39283
I20260812 19:00:35.914399 11586 heartbeater.cc:464] Registering TS with master...
I20260812 19:00:35.914952 11586 heartbeater.cc:511] Master 127.29.112.254:39283 requested a full tablet report, sending...
I20260812 19:00:35.915917 10991 ts_manager.cc:194] Registered new tserver with Master: 7c706ffcb50a44deb89f85769e010e93 (127.29.112.195:45617)
I20260812 19:00:35.916571 10991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:45975
I20260812 19:00:35.923220 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:00:35.955274 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.930258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43608 (local address 127.29.112.254:39283)
0812 19:00:35.930360 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 19:00:35.930364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:00:35.946143 (+ 15779us) server_negotiation.cc:407] Connection header received
0812 19:00:35.946350 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:00:35.946353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:00:35.946402 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:00:35.946454 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:00:35.947425 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.948188 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.949299 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.949599 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.949674 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.951442 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:00:35.951446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:00:35.951448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:00:35.951473 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:00:35.953183 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:00:35.953718 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:00:35.953722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:00:35.953724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:00:35.953814 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:00:35.954181 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:00:35.954184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:00:35.954186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:00:35.954408 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:00:35.954517 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 19:00:35.954955 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 19:00:35.955076 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 19:00:35.959764 10991 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:00:35.963941 10991 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:00:35.991708 11601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.984239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42225 (local address 127.0.0.1:33022)
0812 19:00:35.986175 (+ 1936us) negotiation.cc:107] Waiting for socket to connect
0812 19:00:35.986187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:00:35.986259 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:00:35.986894 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:00:35.986899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:00:35.986939 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:00:35.987083 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.987090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.988544 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.988547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.989938 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.989950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.991187 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.991190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.991296 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.991329 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 19:00:35.991429 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 19:00:35.991534 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1877,"mutex_wait_us":33,"thread_start_us":135,"threads_started":1}
I20260812 19:00:35.992565 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.985408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.29.112.194:42225)
0812 19:00:35.986258 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0812 19:00:35.986276 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:00:35.986303 (+ 27us) server_negotiation.cc:407] Connection header received
0812 19:00:35.986376 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:00:35.986382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:00:35.986584 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:00:35.986736 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:00:35.987226 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.988409 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.990098 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.990876 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.991005 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.991126 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 19:00:35.992293 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0812 19:00:35.992385 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":130,"threads_started":1}
I20260812 19:00:35.995038 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.983673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:45617 (local address 127.0.0.1:33198)
0812 19:00:35.985742 (+ 2069us) negotiation.cc:107] Waiting for socket to connect
0812 19:00:35.985793 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:00:35.985958 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:00:35.987416 (+ 1458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:00:35.987422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:00:35.987445 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:00:35.988713 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.988723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.990836 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.990850 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.993308 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.993329 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.994665 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.994669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.994760 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.994782 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:00:35.994828 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 19:00:35.994908 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1894,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260812 19:00:35.995900 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.984490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33198 (local address 127.29.112.195:45617)
0812 19:00:35.986764 (+ 2274us) server_negotiation.cc:206] Beginning negotiation
0812 19:00:35.986777 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:00:35.986802 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:00:35.986900 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:00:35.986904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:00:35.987217 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:00:35.988942 (+ 1725us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:00:35.989114 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.990595 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.993664 (+ 3069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.994529 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.995513 (+ 984us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.995626 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 19:00:35.995670 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 19:00:35.995749 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2145,"thread_start_us":125,"threads_started":1}
I20260812 19:00:35.995689 11340 tablet_service.cc:1467] Processing CreateTablet for tablet 38e22079b1c746638fa7c18b59f199d6 (DEFAULT_TABLE table=test-table [id=7d7fade18b394e9fb87b20bbe72f5b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:00:35.996788 11340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e22079b1c746638fa7c18b59f199d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:00:35.998714 11498 tablet_service.cc:1467] Processing CreateTablet for tablet 38e22079b1c746638fa7c18b59f199d6 (DEFAULT_TABLE table=test-table [id=7d7fade18b394e9fb87b20bbe72f5b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:00:35.999112 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.984683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:46673 (local address 127.0.0.1:35370)
0812 19:00:35.986428 (+ 1745us) negotiation.cc:107] Waiting for socket to connect
0812 19:00:35.986439 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:00:35.986508 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:00:35.994461 (+ 7953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:00:35.994464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:00:35.994485 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:00:35.994570 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.994580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.996013 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.996018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.997182 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:35.997203 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.998555 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.998561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:35.998761 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.998792 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 19:00:35.998874 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 19:00:35.998948 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1680,"thread_start_us":110,"threads_started":1}
I20260812 19:00:35.999771 11498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e22079b1c746638fa7c18b59f199d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:00:35.999837 11605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:35.985795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35370 (local address 127.29.112.193:46673)
0812 19:00:35.994009 (+ 8214us) server_negotiation.cc:206] Beginning negotiation
0812 19:00:35.994015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:00:35.994037 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:00:35.994107 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:00:35.994112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:00:35.994285 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:00:35.994735 (+ 450us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:00:35.994874 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.995872 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.997349 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:35.998313 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:35.999309 (+ 996us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:35.999466 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 19:00:35.999533 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 19:00:35.999640 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8108,"thread_start_us":76,"threads_started":1}
I20260812 19:00:36.007741 11160 tablet_service.cc:1467] Processing CreateTablet for tablet 38e22079b1c746638fa7c18b59f199d6 (DEFAULT_TABLE table=test-table [id=7d7fade18b394e9fb87b20bbe72f5b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:00:36.008884 11160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e22079b1c746638fa7c18b59f199d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:00:36.013669 11607 tablet_bootstrap.cc:492] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: Bootstrap starting.
I20260812 19:00:36.016613 11607 tablet_bootstrap.cc:654] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: Neither blocks nor log segments found. Creating new log.
I20260812 19:00:36.017668 11607 log.cc:824] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: Log is configured to *not* fsync() on all Append() calls
I20260812 19:00:36.017962 11609 tablet_bootstrap.cc:492] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8: Bootstrap starting.
I20260812 19:00:36.020358 11609 tablet_bootstrap.cc:654] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8: Neither blocks nor log segments found. Creating new log.
I20260812 19:00:36.020562 11610 tablet_bootstrap.cc:492] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a: Bootstrap starting.
I20260812 19:00:36.021951 11609 log.cc:824] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8: Log is configured to *not* fsync() on all Append() calls
I20260812 19:00:36.023350 11610 tablet_bootstrap.cc:654] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a: Neither blocks nor log segments found. Creating new log.
I20260812 19:00:36.024179 11609 tablet_bootstrap.cc:492] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8: No bootstrap required, opened a new log
I20260812 19:00:36.024259 11610 log.cc:824] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a: Log is configured to *not* fsync() on all Append() calls
I20260812 19:00:36.024369 11609 ts_tablet_manager.cc:1397] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260812 19:00:36.026465 11607 tablet_bootstrap.cc:492] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: No bootstrap required, opened a new log
I20260812 19:00:36.026643 11607 ts_tablet_manager.cc:1397] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.002s
I20260812 19:00:36.026659 11610 tablet_bootstrap.cc:492] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a: No bootstrap required, opened a new log
I20260812 19:00:36.026821 11610 ts_tablet_manager.cc:1397] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:00:36.028744 11609 raft_consensus.cc:348] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c706ffcb50a44deb89f85769e010e93" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45617 } } peers { permanent_uuid: "57faad74ee7e43358eed6ea875e3c3a8" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42225 } } peers { permanent_uuid: "e13762bf55f141feab0e244aa6d35d5a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 46673 } }
I20260812 19:00:36.029019 11609 raft_consensus.cc:374] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:00:36.029073 11609 raft_consensus.cc:729] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57faad74ee7e43358eed6ea875e3c3a8, State: Initialized, Role: FOLLOWER
I20260812 19:00:36.029524 11609 consensus_queue.cc:260] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c706ffcb50a44deb89f85769e010e93" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45617 } } peers { permanent_uuid: "57faad74ee7e43358eed6ea875e3c3a8" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42225 } } peers { permanent_uuid: "e13762bf55f141feab0e244aa6d35d5a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 46673 } }
I20260812 19:00:36.030520 11609 ts_tablet_manager.cc:1428] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 19:00:36.030900 11422 heartbeater.cc:503] Master 127.29.112.254:39283 was elected leader, sending a full tablet report...
I20260812 19:00:36.030776 11607 raft_consensus.cc:348] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c706ffcb50a44deb89f85769e010e93" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45617 } } peers { permanent_uuid: "57faad74ee7e43358eed6ea875e3c3a8" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42225 } } peers { permanent_uuid: "e13762bf55f141feab0e244aa6d35d5a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 46673 } }
I20260812 19:00:36.031002 11607 raft_consensus.cc:374] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:00:36.031055 11607 raft_consensus.cc:729] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c706ffcb50a44deb89f85769e010e93, State: Initialized, Role: FOLLOWER
I20260812 19:00:36.031415 11607 consensus_queue.cc:260] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c706ffcb50a44deb89f85769e010e93" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45617 } } peers { permanent_uuid: "57faad74ee7e43358eed6ea875e3c3a8" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42225 } } peers { permanent_uuid: "e13762bf55f141feab0e244aa6d35d5a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 46673 } }
I20260812 19:00:36.032248 11610 raft_consensus.cc:348] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c706ffcb50a44deb89f85769e010e93" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45617 } } peers { permanent_uuid: "57faad74ee7e43358eed6ea875e3c3a8" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42225 } } peers { permanent_uuid: "e13762bf55f141feab0e244aa6d35d5a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 46673 } }
I20260812 19:00:36.032408 11607 ts_tablet_manager.cc:1428] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:00:36.032476 11610 raft_consensus.cc:374] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:00:36.032522 11610 raft_consensus.cc:729] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e13762bf55f141feab0e244aa6d35d5a, State: Initialized, Role: FOLLOWER
I20260812 19:00:36.032923 11610 consensus_queue.cc:260] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c706ffcb50a44deb89f85769e010e93" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45617 } } peers { permanent_uuid: "57faad74ee7e43358eed6ea875e3c3a8" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42225 } } peers { permanent_uuid: "e13762bf55f141feab0e244aa6d35d5a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 46673 } }
I20260812 19:00:36.033035 11586 heartbeater.cc:503] Master 127.29.112.254:39283 was elected leader, sending a full tablet report...
I20260812 19:00:36.034435 11610 ts_tablet_manager.cc:1428] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260812 19:00:36.034785 11249 heartbeater.cc:503] Master 127.29.112.254:39283 was elected leader, sending a full tablet report...
I20260812 19:00:36.139153 11615 raft_consensus.cc:482] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:00:36.139416 11615 raft_consensus.cc:504] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c706ffcb50a44deb89f85769e010e93" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45617 } } peers { permanent_uuid: "57faad74ee7e43358eed6ea875e3c3a8" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42225 } } peers { permanent_uuid: "e13762bf55f141feab0e244aa6d35d5a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 46673 } }
W20260812 19:00:36.139716 11587 tablet.cc:2369] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:00:36.140745 11615 leader_election.cc:288] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57faad74ee7e43358eed6ea875e3c3a8 (127.29.112.194:42225), e13762bf55f141feab0e244aa6d35d5a (127.29.112.193:46673)
I20260812 19:00:36.145736 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:36.140946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42225 (local address 127.29.112.195:39961)
0812 19:00:36.141061 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 19:00:36.141078 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:00:36.141170 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:00:36.141704 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:00:36.141707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:00:36.141727 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:00:36.141867 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:36.141877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:36.142894 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:36.142897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:36.144167 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:36.144178 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:36.145152 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:36.145156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:36.145282 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:36.145320 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 19:00:36.145455 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 19:00:36.145574 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 19:00:36.146160 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:36.141094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:39961 (local address 127.29.112.194:42225)
0812 19:00:36.141339 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 19:00:36.141345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:00:36.141361 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:00:36.141456 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:00:36.141460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:00:36.141510 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:00:36.141581 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:00:36.142008 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:36.142767 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:36.144318 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:36.144998 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:36.145127 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:36.145182 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 19:00:36.145949 (+ 767us) server_negotiation.cc:299] Negotiation successful
0812 19:00:36.146034 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260812 19:00:36.147771 11367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38e22079b1c746638fa7c18b59f199d6" candidate_uuid: "7c706ffcb50a44deb89f85769e010e93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57faad74ee7e43358eed6ea875e3c3a8" is_pre_election: true
I20260812 19:00:36.148128 11367 raft_consensus.cc:2454] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c706ffcb50a44deb89f85769e010e93 in term 0.
I20260812 19:00:36.148661 11450 leader_election.cc:302] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57faad74ee7e43358eed6ea875e3c3a8, 7c706ffcb50a44deb89f85769e010e93; no voters:
I20260812 19:00:36.148945 11615 raft_consensus.cc:2781] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:00:36.149010 11615 raft_consensus.cc:482] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:00:36.149044 11615 raft_consensus.cc:3037] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:00:36.150801 11615 raft_consensus.cc:504] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c706ffcb50a44deb89f85769e010e93" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45617 } } peers { permanent_uuid: "57faad74ee7e43358eed6ea875e3c3a8" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42225 } } peers { permanent_uuid: "e13762bf55f141feab0e244aa6d35d5a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 46673 } }
I20260812 19:00:36.151206 11615 leader_election.cc:288] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [CANDIDATE]: Term 1 election: Requested vote from peers 57faad74ee7e43358eed6ea875e3c3a8 (127.29.112.194:42225), e13762bf55f141feab0e244aa6d35d5a (127.29.112.193:46673)
I20260812 19:00:36.151762 11367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38e22079b1c746638fa7c18b59f199d6" candidate_uuid: "7c706ffcb50a44deb89f85769e010e93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57faad74ee7e43358eed6ea875e3c3a8"
I20260812 19:00:36.151885 11367 raft_consensus.cc:3037] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:00:36.153327 11367 raft_consensus.cc:2454] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c706ffcb50a44deb89f85769e010e93 in term 1.
I20260812 19:00:36.153723 11450 leader_election.cc:302] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57faad74ee7e43358eed6ea875e3c3a8, 7c706ffcb50a44deb89f85769e010e93; no voters:
I20260812 19:00:36.153964 11615 raft_consensus.cc:2781] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:00:36.154213 11615 raft_consensus.cc:686] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 1 LEADER]: Becoming Leader. State: Replica: 7c706ffcb50a44deb89f85769e010e93, State: Running, Role: LEADER
I20260812 19:00:36.154567 11615 consensus_queue.cc:237] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c706ffcb50a44deb89f85769e010e93" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45617 } } peers { permanent_uuid: "57faad74ee7e43358eed6ea875e3c3a8" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42225 } } peers { permanent_uuid: "e13762bf55f141feab0e244aa6d35d5a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 46673 } }
I20260812 19:00:36.155318 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:36.141227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:46673 (local address 127.29.112.195:56473)
0812 19:00:36.145390 (+ 4163us) negotiation.cc:107] Waiting for socket to connect
0812 19:00:36.145416 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 19:00:36.145563 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:00:36.146151 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:00:36.146154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:00:36.146171 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:00:36.146464 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:36.146470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:36.150323 (+ 3853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:36.150327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:36.153889 (+ 3562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:00:36.153902 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:00:36.154888 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:00:36.154892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:00:36.154994 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:36.155019 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 19:00:36.155089 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 19:00:36.155165 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4094,"thread_start_us":97,"threads_started":1}
I20260812 19:00:36.155398 11605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:36.141466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:56473 (local address 127.29.112.193:46673)
0812 19:00:36.141614 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:00:36.141619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:00:36.145493 (+ 3874us) server_negotiation.cc:407] Connection header received
0812 19:00:36.145681 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:00:36.145684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:00:36.145733 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:00:36.145778 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:00:36.146598 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:36.147381 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:36.154041 (+ 6660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:36.154735 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:36.154881 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:36.154946 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 19:00:36.155140 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 19:00:36.155240 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 19:00:36.156543 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38e22079b1c746638fa7c18b59f199d6" candidate_uuid: "7c706ffcb50a44deb89f85769e010e93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e13762bf55f141feab0e244aa6d35d5a" is_pre_election: true
I20260812 19:00:36.156893 11190 raft_consensus.cc:2454] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c706ffcb50a44deb89f85769e010e93 in term 0.
I20260812 19:00:36.157258 11189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38e22079b1c746638fa7c18b59f199d6" candidate_uuid: "7c706ffcb50a44deb89f85769e010e93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e13762bf55f141feab0e244aa6d35d5a"
I20260812 19:00:36.157397 11189 raft_consensus.cc:3037] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:00:36.158243 10990 catalog_manager.cc:5495] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c706ffcb50a44deb89f85769e010e93 (127.29.112.195). New cstate: current_term: 1 leader_uuid: "7c706ffcb50a44deb89f85769e010e93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c706ffcb50a44deb89f85769e010e93" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57faad74ee7e43358eed6ea875e3c3a8" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e13762bf55f141feab0e244aa6d35d5a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 46673 } health_report { overall_health: UNKNOWN } } }
I20260812 19:00:36.158820 11189 raft_consensus.cc:2454] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c706ffcb50a44deb89f85769e010e93 in term 1.
W20260812 19:00:36.168303 11252 tablet.cc:2369] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 19:00:36 dist-test-slave-6nmv krb5kdc[10424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561235, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:00:36.189642 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:36.185090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33206 (local address 127.29.112.195:45617)
0812 19:00:36.185266 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 19:00:36.185270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:00:36.185286 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:00:36.185393 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:00:36.185396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:00:36.185475 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:00:36.185567 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:00:36.185930 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:36.186653 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:36.187944 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:00:36.188226 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:00:36.188377 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:00:36.188735 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:00:36.189182 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:00:36.189352 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 19:00:36.189430 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 19:00:36.189492 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 19:00:36.196636 11367 raft_consensus.cc:1261] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8 [term 1 FOLLOWER]: Refusing update from remote peer 7c706ffcb50a44deb89f85769e010e93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:00:36.196635 11189 raft_consensus.cc:1261] T 38e22079b1c746638fa7c18b59f199d6 P e13762bf55f141feab0e244aa6d35d5a [term 1 FOLLOWER]: Refusing update from remote peer 7c706ffcb50a44deb89f85769e010e93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:00:36.197341 11615 consensus_queue.cc:1035] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57faad74ee7e43358eed6ea875e3c3a8" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:00:36.197561 11621 consensus_queue.cc:1035] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e13762bf55f141feab0e244aa6d35d5a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 46673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:00:36.209194 11625 mvcc.cc:205] Tried to move back new op lower bound from 7317754823453614080 to 7317754823292379136. Current Snapshot: MvccSnapshot[applied={T|T < 7317754823453614080}]
I20260812 19:00:36.220117 10991 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:00:36.220295 10991 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:00:36.222754 10991 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b41ce05fab247d79a98ab2a625c9b2a: Sending DeleteTablet for 3 replicas of tablet 38e22079b1c746638fa7c18b59f199d6
I20260812 19:00:36.223614 11498 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e22079b1c746638fa7c18b59f199d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:00:36 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:33198
I20260812 19:00:36.223760 11340 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e22079b1c746638fa7c18b59f199d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:00:36 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:33022
I20260812 19:00:36.224393 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 11089
I20260812 19:00:36.224440 11634 tablet_replica.cc:330] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: stopping tablet replica
I20260812 19:00:36.224478 11160 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e22079b1c746638fa7c18b59f199d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:00:36 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:35370
I20260812 19:00:36.226904 11634 raft_consensus.cc:2229] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:00:36.227216 11634 raft_consensus.cc:2258] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:00:36.227548 11635 tablet_replica.cc:330] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8: stopping tablet replica
I20260812 19:00:36.227813 11635 raft_consensus.cc:2229] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:00:36.228072 11635 raft_consensus.cc:2258] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:00:36.228654 11634 ts_tablet_manager.cc:1905] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:00:36.230067 11635 ts_tablet_manager.cc:1905] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:00:36.231539 11634 ts_tablet_manager.cc:1918] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:00:36.231642 11634 log.cc:1196] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/38e22079b1c746638fa7c18b59f199d6
I20260812 19:00:36.231959 11634 ts_tablet_manager.cc:1939] T 38e22079b1c746638fa7c18b59f199d6 P 7c706ffcb50a44deb89f85769e010e93: Deleting consensus metadata
I20260812 19:00:36.232995 10974 catalog_manager.cc:4841] TS 7c706ffcb50a44deb89f85769e010e93 (127.29.112.195:45617): tablet 38e22079b1c746638fa7c18b59f199d6 (table test-table [id=7d7fade18b394e9fb87b20bbe72f5b11]) successfully deleted
I20260812 19:00:36.233386 11635 ts_tablet_manager.cc:1918] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:00:36.233465 11635 log.cc:1196] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/38e22079b1c746638fa7c18b59f199d6
I20260812 19:00:36.233783 11635 ts_tablet_manager.cc:1939] T 38e22079b1c746638fa7c18b59f199d6 P 57faad74ee7e43358eed6ea875e3c3a8: Deleting consensus metadata
I20260812 19:00:36.234906 10976 catalog_manager.cc:4841] TS 57faad74ee7e43358eed6ea875e3c3a8 (127.29.112.194:42225): tablet 38e22079b1c746638fa7c18b59f199d6 (table test-table [id=7d7fade18b394e9fb87b20bbe72f5b11]) successfully deleted
W20260812 19:00:36.236842 10976 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:46673 (error 108)
I20260812 19:00:36.237758 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:00:36.237473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:46673 (local address 127.0.0.1:35374)
0812 19:00:36.237591 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 19:00:36.237658 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:46673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 19:00:36.237880 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 11257
W20260812 19:00:36.241294 10976 catalog_manager.cc:4584] TS e13762bf55f141feab0e244aa6d35d5a (127.29.112.193:46673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 38e22079b1c746638fa7c18b59f199d6: Network error: Client connection negotiation failed: client connection to 127.29.112.193:46673: connect: Connection refused (error 111)
I20260812 19:00:36.248085 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 11427
I20260812 19:00:36.257448 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 10953
I20260812 19:00:36.282226 30147 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 19:00:36.637794 30147 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 19:00:36.638015 30147 mini_ranger.cc:66] Stopping Ranger...
I20260812 19:00:36.982489 30147 mini_ranger.cc:68] Stopped Ranger
2026-08-12 19:00:36.982 UTC [10459] LOG: received smart shutdown request
2026-08-12 19:00:36.986 UTC [10459] LOG: background worker "logical replication launcher" (PID 10468) exited with exit code 1
2026-08-12 19:00:36.987 UTC [10463] LOG: shutting down
2026-08-12 19:00:37.128 UTC [10459] LOG: database system is shut down
2026-08-12T19:00:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48461 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:00:37 dist-test-slave-6nmv krb5kdc[11646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:00:37 dist-test-slave-6nmv krb5kdc[11646](info): set up 2 sockets
Aug 12 19:00:37 dist-test-slave-6nmv krb5kdc[11646](info): commencing operation
krb5kdc: starting...
W20260812 19:00:38.310184 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:00:38 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:00:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:00:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.112.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.112.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/rangeradmin_127.29.112.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/rangeradmin_127.29.112.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.112.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.112.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/rangerlookup_127.29.112.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/rangerlookup_127.29.112.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.112.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.112.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/HTTP_127.29.112.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/HTTP_127.29.112.212@KRBTEST.COM.keytab.
I20260812 19:00:38.472517 30147 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmcQeqc/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/postgres -l logfile start
2026-08-12 19:00:40.768 UTC [11687] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 19:00:40.768 UTC [11687] LOG: listening on IPv4 address "127.29.112.212", port 60391
2026-08-12 19:00:40.771 UTC [11687] LOG: listening on Unix socket "/tmp/.s.PGSQL.60391"
2026-08-12 19:00:40.791 UTC [11690] LOG: database system was shut down at 2026-08-12 19:00:39 UTC
2026-08-12 19:00:40.795 UTC [11687] LOG: database system is ready to accept connections
I20260812 19:00:41.795778 30147 mini_postgres.cc:96] Postgres bound to 60391
2026-08-12 19:00:41.803 UTC [11700] FATAL: database "slave" does not exist
127.29.112.212:60391 - accepting connections
I20260812 19:00:41.803690 30147 mini_ranger.cc:161] Starting Ranger...
I20260812 19:00:41.819224 30147 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 19:00:42.256536 30147 mini_ranger.cc:87] Created ranger Postgres database
I20260812 19:00:42.256615 30147 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-admin
2026-08-12 19:00:42,520 [I] DB FLAVOR :POSTGRES
2026-08-12 19:00:42,520 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 19:00:42,520 [I] Checking connection..
2026-08-12 19:00:42,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 19:00:42,758 [I] Checking connection passed.
2026-08-12 19:00:42,758 [I] --------- Verifying version history table ---------
2026-08-12 19:00:42,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 19:00:42,986 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 19:00:42,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 19:00:43,235 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 19:00:43,235 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 19:00:43,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 19:00:43.477 UTC [11773] WARNING: there is no transaction in progress
2026-08-12 19:00:43,484 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 19:00:43,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 19:00:43,764 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 19:00:43,764 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 19:00:43,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 19:00:44,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 19:00:44,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6nmv.c.gcp-upstream.internal','N') ;"
2026-08-12 19:00:44,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 19:00:44,710 [I] Table x_portal_user does not exist in database ranger
2026-08-12 19:00:44,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 19:00:44,950 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 19:00:44,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 19:00:45,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 19:00:45,425 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 19:00:45,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 19:00:46.335 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.350 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.365 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.378 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.395 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.410 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.460 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.466 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.479 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.489 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.498 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.511 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.520 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.526 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.532 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:46.538 UTC [11903] WARNING: there is no transaction in progress
2026-08-12 19:00:47,121 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 19:00:47,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 19:00:47,350 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 19:00:47,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 19:00:47,607 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 19:00:47,608 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 19:00:47,844 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 19:00:47,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 19:00:48,092 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 19:00:48,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6nmv.c.gcp-upstream.internal';"
2026-08-12 19:00:48,585 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 19:00:48,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 19:00:48,814 [I] DB_PATCHES have already been applied
I20260812 19:00:48.820442 30147 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmcQeqc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 19:00:48.820526 30147 mini_ranger.cc:193] Using host: 127.29.112.212
I20260812 19:00:48.824602 30147 mini_ranger.cc:227] Ranger admin URL: http://127.29.112.212:42109
Aug 12, 2026 7:00:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:00:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:00:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 7:00:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 7:00:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 7:00:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 7:00:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 19:00:49.949684 30147 mini_ranger.cc:160] Time spent starting Ranger: real 8.146s user 0.000s sys 0.011s
Aug 12, 2026 7:01:10 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 7:01:10 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 7:01:10 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 7:01:10 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 7:01:11 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 19:01:11.645156 30147 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.112.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.112.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/rangerkms_127.29.112.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.112.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/rangerkms_127.29.112.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.112.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.112.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.112.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/HTTP_127.29.112.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.112.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/HTTP_127.29.112.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 19:01:11 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561271, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 19:01:11.734179 30147 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 19:01:11.745293 30147 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 19:01:12.521778 30147 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 19:01:12.521891 30147 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-kms
I20260812 19:01:12.521914 30147 mini_ranger_kms.cc:227] Using postgres at 127.29.112.212:60391
2026-08-12 19:01:12,705 [I] DB FLAVOR :POSTGRES
2026-08-12 19:01:12,705 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 19:01:12,705 [I] Checking connection
2026-08-12 19:01:12,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 19:01:12,928 [I] connection success
2026-08-12 19:01:12,928 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 19:01:12,928 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 19:01:12,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 19:01:13,157 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 19:01:13,157 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 19:01:13,158 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 19:01:13,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.212:60391/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 19:01:13,393 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 19:01:13.808385 30147 mini_ranger_kms.cc:310] Created kms service
I20260812 19:01:14.005061 30147 mini_ranger_kms.cc:326] Created kudu user
I20260812 19:01:14.047447 30147 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 19:01:14 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561271, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:01:14.449631 30147 mini_ranger_kms.cc:384] Added ranger policy
I20260812 19:01:14.449784 30147 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmcQeqc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmcQeqc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 19:01:14.449903 30147 mini_ranger_kms.cc:240] Using host: 127.29.112.212
I20260812 19:01:14.454479 30147 mini_ranger_kms.cc:276] Ranger KMS PID: 12114
I20260812 19:01:14.454559 30147 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.112.212:43655
Aug 12, 2026 7:01:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:01:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:01:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 7:01:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 7:01:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 7:01:15 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 19:01:16 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786561276, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:16.880468 30147 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.146s user 0.015s sys 0.155s
I20260812 19:01:16.880568 30147 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 19:01:16.880645 30147 mini_ranger_kms.cc:401] 127.29.112.212:43655/kms/v1/keys
Aug 12 19:01:17 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561276, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.212@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
Aug 12 19:01:17 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561276, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.212@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
Aug 12 19:01:19 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:01:19.714984 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:36363
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:36363
--ranger_config_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:43655/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:19.821727 12173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:19.822075 12173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:19.822163 12173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:19.826136 12173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:01:19.826211 12173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:19.826248 12173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:19.826303 12173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:01:19.826339 12173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:01:19.831743 12173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.112.212:43655/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:36363
--ranger_config_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:36363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:19.832741 12173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:19.833796 12173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:19.842901 12180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:19.842970 12173 server_base.cc:1034] running on GCE node
W20260812 19:01:19.843142 12184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:19.842919 12181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:19.843768 12173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:19.845036 12173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:19.846217 12173 hybrid_clock.cc:648] HybridClock initialized: now 1786561279846187 us; error 37 us; skew 500 ppm
Aug 12 19:01:19 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:19.852679 12173 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:01:19.853600 12173 webserver.cc:460] Webserver started at http://127.29.112.254:44959/ using document root <none> and password file <none>
I20260812 19:01:19.854187 12173 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:19.854264 12173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:19.854470 12173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:01:19 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:01:19.899564 12173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "e9459741414841b5ac78e81e03d7a5d3"
format_stamp: "Formatted at 2026-08-12 19:01:19 on dist-test-slave-6nmv"
server_key: "c14ff96fa06e396c4189563ce4ba1ee7"
server_key_iv: "d6e16bce7ec7c319c6980dc869076495"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:19.900174 12173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "e9459741414841b5ac78e81e03d7a5d3"
format_stamp: "Formatted at 2026-08-12 19:01:19 on dist-test-slave-6nmv"
server_key: "c14ff96fa06e396c4189563ce4ba1ee7"
server_key_iv: "d6e16bce7ec7c319c6980dc869076495"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:19.904362 12173 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 19:01:19.917709 12191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:19.919139 12173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:01:19.919260 12173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "e9459741414841b5ac78e81e03d7a5d3"
format_stamp: "Formatted at 2026-08-12 19:01:19 on dist-test-slave-6nmv"
server_key: "c14ff96fa06e396c4189563ce4ba1ee7"
server_key_iv: "d6e16bce7ec7c319c6980dc869076495"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:19.919389 12173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:19.929193 12173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:19.931991 12173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:19.932178 12173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:19.939915 12173 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:36363
I20260812 19:01:19.939950 12250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:36363 every 8 connection(s)
I20260812 19:01:19.940800 12173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:01:19.942564 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 12173
I20260812 19:01:19.942692 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:01:19.942999 30147 external_mini_cluster.cc:1428] Setting key eb65d3458a4413466ba37c16ce9034cd
I20260812 19:01:19.943625 12251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:19.950199 12251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3: Bootstrap starting.
I20260812 19:01:19.952790 12251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3: Neither blocks nor log segments found. Creating new log.
Aug 12 19:01:19 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:19.953691 12251 log.cc:824] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:19.955684 12251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3: No bootstrap required, opened a new log
I20260812 19:01:19.959264 12251 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } }
I20260812 19:01:19.959501 12251 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:19.959589 12251 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9459741414841b5ac78e81e03d7a5d3, State: Initialized, Role: FOLLOWER
I20260812 19:01:19.959844 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:19.944414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48860 (local address 127.29.112.254:36363)
0812 19:01:19.944868 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:19.944886 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:19.944926 (+ 40us) server_negotiation.cc:407] Connection header received
0812 19:01:19.945713 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:19.945733 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:19.946124 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:19.946470 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:19.947315 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:19.948753 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:19.949979 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:19.950636 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:19.950775 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:19.953939 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:19.953945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:19.953960 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:19.953988 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:19.956588 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:19.957117 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:19.957123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:19.957129 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:19.957268 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:19.957565 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:19.957568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:19.957570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:19.958135 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:19.958327 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:19.958628 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 19:01:19.958924 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":142,"threads_started":1}
I20260812 19:01:19.960070 12251 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } }
I20260812 19:01:19.960209 12251 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:01:19.960271 12251 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:01:19.960373 12251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:19.961593 12251 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } }
I20260812 19:01:19.962103 12251 leader_election.cc:302] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9459741414841b5ac78e81e03d7a5d3; no voters:
I20260812 19:01:19.962457 12251 leader_election.cc:288] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:01:19.962512 12257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:01:19.962795 12257 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 1 LEADER]: Becoming Leader. State: Replica: e9459741414841b5ac78e81e03d7a5d3, State: Running, Role: LEADER
I20260812 19:01:19.963141 12257 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } }
I20260812 19:01:19.963737 12251 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:01:19.964824 12258 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9459741414841b5ac78e81e03d7a5d3. Latest consensus state: current_term: 1 leader_uuid: "e9459741414841b5ac78e81e03d7a5d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } } }
I20260812 19:01:19.964941 12258 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:19.965210 12259 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9459741414841b5ac78e81e03d7a5d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } } }
I20260812 19:01:19.965348 12259 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:19.965351 12267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:01:19.969398 12267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:01:19.969947 12251 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 19:01:19.982793 12267 catalog_manager.cc:1347] Generated new cluster ID: 3f1760b5b0f4435590708996353e09ab
I20260812 19:01:19.982900 12267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:01:20.005401 12267 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:01:20.007248 12267 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:01:20.013466 12267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3: Generated new TSK 0
I20260812 19:01:20.014433 12267 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 19:01:20.621111 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:43655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:36363
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:20.759912 12302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:20.760284 12302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:20.760350 12302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:20.766342 12302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:20.766419 12302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:20.766499 12302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:01:20.774080 12302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.112.212:43655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:36363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:20.775393 12302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:20.777115 12302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:20.789080 12312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:20.789131 12314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:20.789546 12311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:20.790010 12302 server_base.cc:1034] running on GCE node
I20260812 19:01:20.790418 12302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:20.791126 12302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:20.792315 12302 hybrid_clock.cc:648] HybridClock initialized: now 1786561280792287 us; error 40 us; skew 500 ppm
Aug 12 19:01:20 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561280, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:20.795830 12302 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:01:20.796759 12302 webserver.cc:460] Webserver started at http://127.29.112.193:36241/ using document root <none> and password file <none>
I20260812 19:01:20.797303 12302 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:20.797389 12302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:20.797668 12302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:01:20 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561280, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:01:20.816836 12302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "b0b83e9efa5443f5af45b1444715d0a3"
format_stamp: "Formatted at 2026-08-12 19:01:20 on dist-test-slave-6nmv"
server_key: "b35eef41d0e9aba7597bd3916d02e4d4"
server_key_iv: "d1417bd05010f01692c39e21d1b4bbc2"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:20.817718 12302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "b0b83e9efa5443f5af45b1444715d0a3"
format_stamp: "Formatted at 2026-08-12 19:01:20 on dist-test-slave-6nmv"
server_key: "b35eef41d0e9aba7597bd3916d02e4d4"
server_key_iv: "d1417bd05010f01692c39e21d1b4bbc2"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:20.822631 12302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:01:20.855152 12323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:20.856637 12302 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260812 19:01:20.856801 12302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "b0b83e9efa5443f5af45b1444715d0a3"
format_stamp: "Formatted at 2026-08-12 19:01:20 on dist-test-slave-6nmv"
server_key: "b35eef41d0e9aba7597bd3916d02e4d4"
server_key_iv: "d1417bd05010f01692c39e21d1b4bbc2"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:20.856951 12302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:20.883141 12302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:20.887086 12302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:20.887355 12302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:20.888196 12302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:20.889302 12302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:20.889412 12302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:20.889556 12302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:20.889647 12302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:20.901389 12302 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:38445
I20260812 19:01:20.901551 12446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:38445 every 8 connection(s)
I20260812 19:01:20.902696 12302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:01:20.905866 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 12302
I20260812 19:01:20.905988 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 19:01:20.906281 30147 external_mini_cluster.cc:1428] Setting key 9974c56bfac3818d7351f9bb4728cefe
Aug 12 19:01:20 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561280, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
I20260812 19:01:20.923444 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:20.904837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:50497 (local address 127.29.112.254:36363)
0812 19:01:20.905152 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:20.905160 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:20.907130 (+ 1970us) server_negotiation.cc:407] Connection header received
0812 19:01:20.910479 (+ 3349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:20.910482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:20.910536 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:20.910600 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:20.912367 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:20.913244 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:20.914915 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:20.915296 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:20.915381 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:20.918659 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:20.918663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:20.918666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:20.918693 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:20.920693 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:20.921390 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:20.921394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:20.921395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:20.921529 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:20.922052 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:20.922056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:20.922057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:20.922280 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:20.922381 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:20.923086 (+ 705us) server_negotiation.cc:299] Negotiation successful
0812 19:01:20.923224 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":127,"threads_started":1}
I20260812 19:01:20.925036 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:20.905291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:36363 (local address 127.29.112.193:50497)
0812 19:01:20.906975 (+ 1684us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:20.907032 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:20.910222 (+ 3190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:20.911022 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:20.911032 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:20.911401 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:20.912187 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:20.912209 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:20.913395 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:20.913399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:20.914769 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:20.914780 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:20.915613 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:20.915618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:20.915813 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:20.916556 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:20.916578 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:20.918461 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:20.920815 (+ 2354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:20.920821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:20.920837 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:20.921196 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:20.921654 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:20.921656 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:20.921659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:20.921848 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:20.922447 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:20.922454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:20.922790 (+ 336us) client_negotiation.cc:769] Sending connection context
0812 19:01:20.923633 (+ 843us) client_negotiation.cc:240] Negotiation successful
0812 19:01:20.923978 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1471,"thread_start_us":151,"threads_started":1}
I20260812 19:01:20.926648 12447 heartbeater.cc:347] Connected to a master server at 127.29.112.254:36363
I20260812 19:01:20.926903 12447 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:20.927491 12447 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
I20260812 19:01:20.929351 12212 ts_manager.cc:194] Registered new tserver with Master: b0b83e9efa5443f5af45b1444715d0a3 (127.29.112.193:38445)
I20260812 19:01:20.930928 12212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:50497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 19:01:20.945382 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:43655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:36363
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:01:20 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786561280, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 19:01:21.104678 12453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:21.105006 12453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:21.105067 12453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:21.110900 12453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:21.111049 12453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:21.111181 12453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:01:21.118664 12453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.112.212:43655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:36363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:21.120029 12453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:21.121394 12453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:21.135344 12460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:21.135476 12461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:21.136178 12463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:21.136737 12453 server_base.cc:1034] running on GCE node
I20260812 19:01:21.137291 12453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:21.138128 12453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:21.139328 12453 hybrid_clock.cc:648] HybridClock initialized: now 1786561281139297 us; error 46 us; skew 500 ppm
Aug 12 19:01:21 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561281, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:21.142956 12453 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:01:21.144137 12453 webserver.cc:460] Webserver started at http://127.29.112.194:34799/ using document root <none> and password file <none>
I20260812 19:01:21.144816 12453 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:21.144918 12453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:21.145193 12453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:01:21 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561281, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:01:21.165316 12453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "c438ebd25782425c9e0f3ce42ccdae8c"
format_stamp: "Formatted at 2026-08-12 19:01:21 on dist-test-slave-6nmv"
server_key: "b69d1563921b84d80f79ad50e0d4747e"
server_key_iv: "4ec07ace8545a6db630bd7bc88265639"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:21.166145 12453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "c438ebd25782425c9e0f3ce42ccdae8c"
format_stamp: "Formatted at 2026-08-12 19:01:21 on dist-test-slave-6nmv"
server_key: "b69d1563921b84d80f79ad50e0d4747e"
server_key_iv: "4ec07ace8545a6db630bd7bc88265639"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:21.171679 12453 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 19:01:21.189072 12470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:21.190804 12453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:01:21.190987 12453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "c438ebd25782425c9e0f3ce42ccdae8c"
format_stamp: "Formatted at 2026-08-12 19:01:21 on dist-test-slave-6nmv"
server_key: "b69d1563921b84d80f79ad50e0d4747e"
server_key_iv: "4ec07ace8545a6db630bd7bc88265639"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:21.191144 12453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:21.218744 12453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:21.221932 12453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:21.222115 12453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:21.222777 12453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:21.223780 12453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:21.223842 12453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:21.223902 12453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:21.223930 12453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:21.234913 12453 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:39829
I20260812 19:01:21.234961 12610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:39829 every 8 connection(s)
I20260812 19:01:21.236006 12453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
Aug 12 19:01:21 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561281, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:21.245703 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 12453
I20260812 19:01:21.245860 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 19:01:21.246171 30147 external_mini_cluster.cc:1428] Setting key 9cb73f49b831aef22553877acafe5e54
I20260812 19:01:21.255458 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:21.237928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:53161 (local address 127.29.112.254:36363)
0812 19:01:21.238056 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:21.238060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:21.239259 (+ 1199us) server_negotiation.cc:407] Connection header received
0812 19:01:21.240146 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:21.240150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:21.240209 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:21.240302 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:21.241355 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:21.241935 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:21.243028 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:21.243254 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:21.243360 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:21.250728 (+ 7368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:21.250732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:21.250735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:21.250763 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:21.252759 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:21.253467 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:21.253470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:21.253472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:21.253574 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:21.254066 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:21.254070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:21.254071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:21.254288 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:21.254358 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:21.255104 (+ 746us) server_negotiation.cc:299] Negotiation successful
0812 19:01:21.255241 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 19:01:21.257148 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:21.238583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:36363 (local address 127.29.112.194:53161)
0812 19:01:21.239112 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:21.239150 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:21.239939 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:21.240483 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:21.240489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:21.240755 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:21.241209 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:21.241221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:21.242069 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:21.242073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:21.242896 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:21.242905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:21.243399 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:21.243403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:21.243570 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:21.244163 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:21.244178 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:21.250512 (+ 6334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:21.252932 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:21.252938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:21.252956 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:21.253343 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:21.253694 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:21.253697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:21.253699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:21.253943 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:21.254461 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:21.254467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:21.254813 (+ 346us) client_negotiation.cc:769] Sending connection context
0812 19:01:21.255719 (+ 906us) client_negotiation.cc:240] Negotiation successful
0812 19:01:21.256084 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":191,"threads_started":1}
I20260812 19:01:21.258661 12611 heartbeater.cc:347] Connected to a master server at 127.29.112.254:36363
I20260812 19:01:21.258948 12611 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:21.259593 12611 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
I20260812 19:01:21.260876 12212 ts_manager.cc:194] Registered new tserver with Master: c438ebd25782425c9e0f3ce42ccdae8c (127.29.112.194:39829)
I20260812 19:01:21.261924 12212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:53161
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 19:01:21.285368 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:43655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:36363
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:01:21 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561280, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
W20260812 19:01:21.426234 12616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:21.426649 12616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:21.426757 12616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:21.432442 12616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:21.432588 12616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:21.432732 12616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:01:21.440554 12616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.112.212:43655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:36363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:21.442106 12616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:21.443441 12616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:01:21.456627 12616 server_base.cc:1034] running on GCE node
W20260812 19:01:21.457098 12625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:21.457449 12623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:21.457712 12621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:21.458235 12616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:21.459003 12616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:21.460170 12616 hybrid_clock.cc:648] HybridClock initialized: now 1786561281460084 us; error 101 us; skew 500 ppm
Aug 12 19:01:21 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561281, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:21.464099 12616 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:01:21.465150 12616 webserver.cc:460] Webserver started at http://127.29.112.195:39039/ using document root <none> and password file <none>
I20260812 19:01:21.465854 12616 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:21.465934 12616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:21.466150 12616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 19:01:21 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561281, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:01:21.502661 12616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "c7aff0658cb140b1abf27c64d3c80463"
format_stamp: "Formatted at 2026-08-12 19:01:21 on dist-test-slave-6nmv"
server_key: "4e1f3118319ccd3f5cecca85dce33c4c"
server_key_iv: "9bb776e2872d1c67f036a1dc4a8cbff0"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:21.503628 12616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "c7aff0658cb140b1abf27c64d3c80463"
format_stamp: "Formatted at 2026-08-12 19:01:21 on dist-test-slave-6nmv"
server_key: "4e1f3118319ccd3f5cecca85dce33c4c"
server_key_iv: "9bb776e2872d1c67f036a1dc4a8cbff0"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:21.508878 12616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 19:01:21.534373 12633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:21.536096 12616 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260812 19:01:21.536228 12616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "c7aff0658cb140b1abf27c64d3c80463"
format_stamp: "Formatted at 2026-08-12 19:01:21 on dist-test-slave-6nmv"
server_key: "4e1f3118319ccd3f5cecca85dce33c4c"
server_key_iv: "9bb776e2872d1c67f036a1dc4a8cbff0"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:21.536353 12616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:21.560683 12616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:21.564564 12616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:21.564831 12616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:21.565518 12616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:21.566679 12616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:21.566790 12616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:21.566887 12616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:21.566953 12616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:21.580199 12616 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:45515
I20260812 19:01:21.581517 12616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:01:21.584872 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 12616
I20260812 19:01:21.584990 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 19:01:21.585291 30147 external_mini_cluster.cc:1428] Setting key 64351b321bb6e71576c6e0aff6c91666
I20260812 19:01:21.588508 12764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:45515 every 8 connection(s)
Aug 12 19:01:21 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561281, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:21.610671 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:21.592153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:50099 (local address 127.29.112.254:36363)
0812 19:01:21.592284 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:21.592290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:21.600302 (+ 8012us) server_negotiation.cc:407] Connection header received
0812 19:01:21.601260 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:21.601263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:21.601329 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:21.601429 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:21.602599 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:21.603115 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:21.604215 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:21.604448 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:21.604562 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:21.606944 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:21.606947 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:21.606949 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:21.607004 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:21.608482 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:21.608989 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:21.608992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:21.608994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:21.609085 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:21.609441 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:21.609444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:21.609446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:21.609616 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:21.609723 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:21.610383 (+ 660us) server_negotiation.cc:299] Negotiation successful
0812 19:01:21.610502 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:01:21.612210 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:21.590867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:36363 (local address 127.29.112.195:50099)
0812 19:01:21.600151 (+ 9284us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:21.600208 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:21.601067 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:21.601602 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:21.601608 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:21.601964 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:21.602458 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:21.602471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:21.603261 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:21.603264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:21.604094 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:21.604102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:21.604606 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:21.604610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:21.604773 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:21.605402 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:21.605418 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:21.606779 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:21.608613 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:21.608619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:21.608633 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:21.608884 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:21.609185 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:21.609188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:21.609190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:21.609328 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:21.609777 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:21.609783 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:21.610114 (+ 331us) client_negotiation.cc:769] Sending connection context
0812 19:01:21.611125 (+ 1011us) client_negotiation.cc:240] Negotiation successful
0812 19:01:21.611424 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8968,"thread_start_us":204,"threads_started":1}
I20260812 19:01:21.613308 12765 heartbeater.cc:347] Connected to a master server at 127.29.112.254:36363
I20260812 19:01:21.613543 12765 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:21.614061 12765 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
I20260812 19:01:21.614987 12212 ts_manager.cc:194] Registered new tserver with Master: c7aff0658cb140b1abf27c64d3c80463 (127.29.112.195:45515)
I20260812 19:01:21.615545 12212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:50099
I20260812 19:01:21.622725 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:01:21.623484 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 12302
I20260812 19:01:21.639350 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 12453
I20260812 19:01:21.649587 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 12616
I20260812 19:01:21.660801 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 12173
I20260812 19:01:21.674261 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:36363
--webserver_interface=127.29.112.254
--webserver_port=44959
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:36363
--ranger_config_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:43655/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.112.212:43655/kms with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:21.781881 12768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:21.782148 12768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:21.782184 12768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:21.786099 12768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:01:21.786151 12768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:21.786170 12768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:21.786202 12768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:01:21.786237 12768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:01:21.791417 12768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.112.212:43655/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:36363
--ranger_config_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:36363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=44959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:21.792404 12768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:21.793382 12768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:21.801473 12778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:21.801573 12768 server_base.cc:1034] running on GCE node
W20260812 19:01:21.801492 12773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:21.801481 12774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:21.802266 12768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:21.803252 12768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:21.804404 12768 hybrid_clock.cc:648] HybridClock initialized: now 1786561281804388 us; error 29 us; skew 500 ppm
Aug 12 19:01:21 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561281, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:21.806994 12768 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:01:21.807787 12768 webserver.cc:460] Webserver started at http://127.29.112.254:44959/ using document root <none> and password file <none>
I20260812 19:01:21.808312 12768 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:21.808403 12768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 19:01:21 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561281, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:01:21.854206 12768 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:01:21.856284 12787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:21.857524 12768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:01:21.857676 12768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "e9459741414841b5ac78e81e03d7a5d3"
format_stamp: "Formatted at 2026-08-12 19:01:19 on dist-test-slave-6nmv"
server_key: "c14ff96fa06e396c4189563ce4ba1ee7"
server_key_iv: "d6e16bce7ec7c319c6980dc869076495"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:21.858287 12768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:21.873044 12768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:21.875380 12768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:21.875558 12768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:21.882087 12768 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:36363
I20260812 19:01:21.882154 12851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:36363 every 8 connection(s)
I20260812 19:01:21.883159 12768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:01:21.887120 12852 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:01:21.888695 12852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3: Bootstrap starting.
I20260812 19:01:21.892141 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 12768
I20260812 19:01:21.892640 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:43655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:38445
--local_ip_for_outbound_sockets=127.29.112.193
--tserver_master_addrs=127.29.112.254:36363
--webserver_port=36241
--webserver_interface=127.29.112.193
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.112.212:43655/kms with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:01:21.906255 12852 log.cc:824] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:21.913434 12852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:01:21.913908 12852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3: Bootstrap complete.
I20260812 19:01:21.918081 12852 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } }
I20260812 19:01:21.918481 12852 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9459741414841b5ac78e81e03d7a5d3, State: Initialized, Role: FOLLOWER
I20260812 19:01:21.919010 12852 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } }
I20260812 19:01:21.919155 12852 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:01:21.919217 12852 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:01:21.919320 12852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:01:21.921491 12852 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } }
I20260812 19:01:21.921857 12852 leader_election.cc:302] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9459741414841b5ac78e81e03d7a5d3; no voters:
I20260812 19:01:21.922207 12852 leader_election.cc:288] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 19:01:21.922290 12856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:01:21.922493 12856 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [term 2 LEADER]: Becoming Leader. State: Replica: e9459741414841b5ac78e81e03d7a5d3, State: Running, Role: LEADER
I20260812 19:01:21.922847 12856 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } }
I20260812 19:01:21.923424 12852 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:01:21.923816 12856 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9459741414841b5ac78e81e03d7a5d3. Latest consensus state: current_term: 2 leader_uuid: "e9459741414841b5ac78e81e03d7a5d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } } }
I20260812 19:01:21.923924 12856 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:21.924350 12857 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e9459741414841b5ac78e81e03d7a5d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9459741414841b5ac78e81e03d7a5d3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 36363 } } }
I20260812 19:01:21.924512 12857 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:21.924906 12860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:01:21.928198 12860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:01:21.928753 12852 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 19:01:21.928836 12860 catalog_manager.cc:1259] Loaded cluster ID: 3f1760b5b0f4435590708996353e09ab
I20260812 19:01:21.928891 12860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:01:21.931283 12860 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:01:21.931953 12860 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3: Loaded TSK: 0
I20260812 19:01:21.933131 12860 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 19:01:22.025851 12854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:22.026178 12854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:22.026238 12854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:22.031939 12854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:22.032015 12854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:22.032104 12854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:01:22.039471 12854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.112.212:43655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:38445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=36241
--tserver_master_addrs=127.29.112.254:36363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:22.040807 12854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:22.042132 12854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:22.054047 12897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:22.054160 12894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:22.054061 12895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:22.055172 12854 server_base.cc:1034] running on GCE node
I20260812 19:01:22.055610 12854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:22.056329 12854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:22.057551 12854 hybrid_clock.cc:648] HybridClock initialized: now 1786561282057534 us; error 34 us; skew 500 ppm
Aug 12 19:01:22 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561282, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:22.061177 12854 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:01:22.062309 12854 webserver.cc:460] Webserver started at http://127.29.112.193:36241/ using document root <none> and password file <none>
I20260812 19:01:22.063016 12854 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:22.063123 12854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 19:01:22 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561282, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:01:22.089648 12854 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:01:22.093024 12907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:22.094585 12854 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 19:01:22.094740 12854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "b0b83e9efa5443f5af45b1444715d0a3"
format_stamp: "Formatted at 2026-08-12 19:01:20 on dist-test-slave-6nmv"
server_key: "b35eef41d0e9aba7597bd3916d02e4d4"
server_key_iv: "d1417bd05010f01692c39e21d1b4bbc2"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:22.095449 12854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:22.115248 12854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:22.118602 12854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:22.118813 12854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:22.119576 12854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:22.120608 12854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:22.120678 12854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:22.120750 12854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:22.120779 12854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:22.131719 12854 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:38445
I20260812 19:01:22.132488 13032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:38445 every 8 connection(s)
I20260812 19:01:22.132947 12854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:01:22.141458 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 12854
I20260812 19:01:22.141943 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:43655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:39829
--local_ip_for_outbound_sockets=127.29.112.194
--tserver_master_addrs=127.29.112.254:36363
--webserver_port=34799
--webserver_interface=127.29.112.194
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.112.212:43655/kms with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:01:22 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561282, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:22.165087 13036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:22.135562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:36363 (local address 127.29.112.193:59797)
0812 19:01:22.136775 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:22.136831 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:22.137990 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:22.140281 (+ 2291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:22.140290 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:22.140673 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:22.141319 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:22.141339 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.143134 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.143137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:22.144301 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:22.144315 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.145375 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.145382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:22.145625 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:22.146479 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:22.146509 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:22.152703 (+ 6194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:22.160996 (+ 8293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:22.161004 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:22.161020 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:22.161476 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:22.161900 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:22.161904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:22.161908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:22.162141 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:22.162985 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:22.162995 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:22.163368 (+ 373us) client_negotiation.cc:769] Sending connection context
0812 19:01:22.163621 (+ 253us) client_negotiation.cc:240] Negotiation successful
0812 19:01:22.163984 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"thread_start_us":172,"threads_started":1}
I20260812 19:01:22.166893 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:22.135421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:59797 (local address 127.29.112.254:36363)
0812 19:01:22.138528 (+ 3107us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:22.138552 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:22.138598 (+ 46us) server_negotiation.cc:407] Connection header received
0812 19:01:22.138892 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:22.138906 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:22.139345 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:22.139766 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:22.141558 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.142986 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.144451 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.145188 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.145370 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:22.153012 (+ 7642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:22.153020 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:22.153039 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:22.153076 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:22.160797 (+ 7721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:22.161625 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:22.161631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:22.161637 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:22.161794 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:22.162271 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:22.162275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:22.162277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:22.162815 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:22.162941 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:22.165660 (+ 2719us) server_negotiation.cc:299] Negotiation successful
0812 19:01:22.165989 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2937,"thread_start_us":117,"threads_started":1}
I20260812 19:01:22.169713 13033 heartbeater.cc:347] Connected to a master server at 127.29.112.254:36363
I20260812 19:01:22.169986 13033 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:22.170547 13033 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
W20260812 19:01:22.172698 13033 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:36363 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 19:01:22.172837 13033 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:22.173017 13033 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
I20260812 19:01:22.173746 13033 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:22.173928 13033 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
W20260812 19:01:22.174566 13033 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:01:22.304984 13037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:22.305311 13037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:22.305382 13037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:22.311292 13037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:22.311368 13037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:22.311456 13037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:01:22.318949 13037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.112.212:43655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:39829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=34799
--tserver_master_addrs=127.29.112.254:36363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:22.320217 13037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:22.321532 13037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:22.333889 13046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:22.333884 13044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:22.333911 13043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:22.334619 13037 server_base.cc:1034] running on GCE node
I20260812 19:01:22.335012 13037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:22.335584 13037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:22.336777 13037 hybrid_clock.cc:648] HybridClock initialized: now 1786561282336746 us; error 48 us; skew 500 ppm
Aug 12 19:01:22 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561282, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:22.339540 13037 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:01:22.340494 13037 webserver.cc:460] Webserver started at http://127.29.112.194:34799/ using document root <none> and password file <none>
I20260812 19:01:22.341056 13037 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:22.341158 13037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 19:01:22 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561282, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:01:22.365536 13037 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.003s sys 0.000s
I20260812 19:01:22.368808 13056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:22.370952 13037 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260812 19:01:22.371083 13037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "c438ebd25782425c9e0f3ce42ccdae8c"
format_stamp: "Formatted at 2026-08-12 19:01:21 on dist-test-slave-6nmv"
server_key: "b69d1563921b84d80f79ad50e0d4747e"
server_key_iv: "4ec07ace8545a6db630bd7bc88265639"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:22.371780 13037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:22.404345 13037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:22.407891 13037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:22.408174 13037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:22.408916 13037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:22.409898 13037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:22.409960 13037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:22.410037 13037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:22.410071 13037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:22.421185 13037 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:39829
I20260812 19:01:22.422165 13189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:39829 every 8 connection(s)
I20260812 19:01:22.422420 13037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:01:22.431104 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 13037
I20260812 19:01:22.431665 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.212:43655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:45515
--local_ip_for_outbound_sockets=127.29.112.195
--tserver_master_addrs=127.29.112.254:36363
--webserver_port=39039
--webserver_interface=127.29.112.195
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.112.212:43655/kms with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:01:22 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561282, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:22.438784 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:22.424551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:50719 (local address 127.29.112.254:36363)
0812 19:01:22.424712 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:22.424717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:22.425619 (+ 902us) server_negotiation.cc:407] Connection header received
0812 19:01:22.426698 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:22.426701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:22.426756 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:22.426832 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:22.427817 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.428507 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.429577 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.429799 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.429937 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:22.434665 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:22.434670 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:22.434674 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:22.434706 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:22.436154 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:22.436742 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:22.436746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:22.436747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:22.436923 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:22.437379 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:22.437383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:22.437385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:22.437635 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:22.437725 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:22.438466 (+ 741us) server_negotiation.cc:299] Negotiation successful
0812 19:01:22.438620 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 19:01:22.440357 13192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:22.425035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:36363 (local address 127.29.112.194:50719)
0812 19:01:22.425475 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:22.425525 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:22.426504 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:22.426989 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:22.426995 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:22.427262 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:22.427683 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:22.427694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.428637 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.428640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:22.429461 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:22.429469 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.430115 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.430124 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:22.430258 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:22.430902 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:22.430925 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:22.434487 (+ 3562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:22.436287 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:22.436293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:22.436307 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:22.436606 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:22.437057 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:22.437060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:22.437062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:22.437256 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:22.437827 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:22.437861 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:22.438171 (+ 310us) client_negotiation.cc:769] Sending connection context
0812 19:01:22.438969 (+ 798us) client_negotiation.cc:240] Negotiation successful
0812 19:01:22.439321 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":119,"threads_started":1}
I20260812 19:01:22.441735 13190 heartbeater.cc:347] Connected to a master server at 127.29.112.254:36363
I20260812 19:01:22.442026 13190 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:22.442628 13190 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
W20260812 19:01:22.444291 13190 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:36363 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 19:01:22.444419 13190 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:22.444564 13190 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
I20260812 19:01:22.445334 13190 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:22.445511 13190 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
W20260812 19:01:22.446352 13190 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:01:22.587937 13194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:22.588271 13194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:22.588351 13194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:22.594197 13194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:22.594270 13194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:22.594352 13194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:01:22.606371 13194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.112.212:43655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:45515
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=39039
--tserver_master_addrs=127.29.112.254:36363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:22.607635 13194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:22.608937 13194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:22.620954 13206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:22.621105 13194 server_base.cc:1034] running on GCE node
W20260812 19:01:22.621305 13204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:22.620985 13203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:22.621856 13194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:22.622648 13194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:22.625938 13194 hybrid_clock.cc:648] HybridClock initialized: now 1786561282625884 us; error 64 us; skew 500 ppm
Aug 12 19:01:22 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561282, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:22.629146 13194 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:01:22.630301 13194 webserver.cc:460] Webserver started at http://127.29.112.195:39039/ using document root <none> and password file <none>
I20260812 19:01:22.631014 13194 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:22.631104 13194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 19:01:22 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561282, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:01:22.660238 13194 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:01:22.663018 13216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:22.664358 13194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:01:22.664491 13194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "c7aff0658cb140b1abf27c64d3c80463"
format_stamp: "Formatted at 2026-08-12 19:01:21 on dist-test-slave-6nmv"
server_key: "4e1f3118319ccd3f5cecca85dce33c4c"
server_key_iv: "9bb776e2872d1c67f036a1dc4a8cbff0"
server_key_version: "kuduclusterkey@0"
I20260812 19:01:22.665040 13194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:22.687448 13194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:22.690721 13194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:22.690898 13194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:22.691509 13194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:22.692340 13194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:22.692408 13194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:22.692485 13194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:22.692533 13194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:22.702769 13194 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:45515
I20260812 19:01:22.702911 13348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:45515 every 8 connection(s)
I20260812 19:01:22.703734 13194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:01:22.710620 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 13194
Aug 12 19:01:22 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561282, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:22.717762 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:22.705900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:36363 (local address 127.29.112.195:44143)
0812 19:01:22.706337 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:22.706372 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:22.707136 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:22.707613 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:22.707620 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:22.707930 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:22.708599 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:22.708620 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.709641 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.709643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:22.710654 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:22.710665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.711366 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.711369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:22.711556 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:22.712230 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:22.712252 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:22.713719 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:22.715294 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:22.715299 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:22.715310 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:22.715598 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:22.715865 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:22.715869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:22.715871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:22.716038 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:22.716449 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:22.716454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:22.716721 (+ 267us) client_negotiation.cc:769] Sending connection context
0812 19:01:22.716905 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 19:01:22.717130 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":130,"threads_started":1}
I20260812 19:01:22.718216 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:22.705450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:44143 (local address 127.29.112.254:36363)
0812 19:01:22.705571 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:22.705575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:22.706468 (+ 893us) server_negotiation.cc:407] Connection header received
0812 19:01:22.707318 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:22.707321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:22.707376 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:22.707446 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:22.708787 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.709501 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.710850 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.711097 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.711185 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:22.713873 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:22.713876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:22.713879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:22.713902 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:22.715189 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:22.715682 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:22.715686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:22.715687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:22.715777 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:22.716143 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:22.716147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:22.716149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:22.716326 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:22.716404 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:22.717971 (+ 1567us) server_negotiation.cc:299] Negotiation successful
0812 19:01:22.718107 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 19:01:22.719614 13349 heartbeater.cc:347] Connected to a master server at 127.29.112.254:36363
I20260812 19:01:22.719836 13349 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:22.720264 13349 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
I20260812 19:01:22.721602 12812 ts_manager.cc:194] Registered new tserver with Master: c7aff0658cb140b1abf27c64d3c80463 (127.29.112.195:45515)
I20260812 19:01:22.722872 12812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:44143
I20260812 19:01:22.726857 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:22.711304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48866 (local address 127.29.112.254:36363)
0812 19:01:22.711948 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:22.711956 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:22.711966 (+ 10us) server_negotiation.cc:407] Connection header received
0812 19:01:22.711994 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:22.711995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:22.712065 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:22.712153 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:22.714194 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.715727 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.718712 (+ 2985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:22.719046 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:22.719146 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:22.723787 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:22.723790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:22.723792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:22.723813 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:22.725228 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:22.725686 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:22.725691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:22.725692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:22.725769 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:22.726061 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:22.726064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:22.726065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:22.726238 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:22.726405 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:22.726602 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 19:01:22.726706 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":116,"threads_started":1}
Aug 12 19:01:23 dist-test-slave-6nmv krb5kdc[11646](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786561283, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:23.174819 13033 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:23.175196 13033 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
I20260812 19:01:23.175935 12812 ts_manager.cc:194] Registered new tserver with Master: b0b83e9efa5443f5af45b1444715d0a3 (127.29.112.193:38445)
I20260812 19:01:23.176721 12812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:59797
Aug 12 19:01:23 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561283, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
I20260812 19:01:23.446587 13190 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:23.446918 13190 heartbeater.cc:511] Master 127.29.112.254:36363 requested a full tablet report, sending...
I20260812 19:01:23.447716 12812 ts_manager.cc:194] Registered new tserver with Master: c438ebd25782425c9e0f3ce42ccdae8c (127.29.112.194:39829)
I20260812 19:01:23.448489 12812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:50719
I20260812 19:01:23.454016 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:01:23.465931 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.456076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.29.112.254:36363)
0812 19:01:23.456998 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:23.457003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:23.457020 (+ 17us) server_negotiation.cc:407] Connection header received
0812 19:01:23.457061 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:23.457064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:23.457112 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:23.458140 (+ 1028us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:23.458238 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.459041 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.460163 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.460504 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.460623 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.462224 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:23.462228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:23.462231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:23.462255 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:23.464056 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:23.464494 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:23.464497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:23.464499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:23.464597 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:23.464972 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:23.464977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:23.464979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:23.465202 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:23.465339 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:23.465568 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 19:01:23.465710 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":844,"thread_start_us":71,"threads_started":1}
I20260812 19:01:23.470567 12812 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:01:23.475359 12812 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:01:23.496814 13383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.488713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:39829 (local address 127.0.0.1:39998)
0812 19:01:23.489397 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:23.489449 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:23.489581 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:23.491709 (+ 2128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:23.491712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:23.491730 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:23.491826 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:23.491832 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.493212 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.493215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:23.494867 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:23.494878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.496273 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.496275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:23.496352 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.496379 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 19:01:23.496507 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 19:01:23.496678 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":52,"thread_start_us":125,"threads_started":1}
I20260812 19:01:23.496815 13388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.490284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39998 (local address 127.29.112.194:39829)
0812 19:01:23.491089 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:23.491096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:23.491118 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:01:23.491190 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:23.491195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:23.491380 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:23.491523 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:23.491965 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.493078 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.495117 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.495919 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.496047 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.496208 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:23.496542 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 19:01:23.496651 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":109,"threads_started":1}
I20260812 19:01:23.498095 13385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.489000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:38445 (local address 127.0.0.1:48500)
0812 19:01:23.490742 (+ 1742us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:23.490756 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:23.490858 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:23.491496 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:23.491501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:23.491523 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:23.491679 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:23.491686 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.493868 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.493872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:23.495068 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:23.495082 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.496314 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.496317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:23.496409 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.496431 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:01:23.496498 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 19:01:23.496675 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1678,"mutex_wait_us":3,"thread_start_us":110,"threads_started":1}
I20260812 19:01:23.498553 13384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.489172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48500 (local address 127.29.112.193:38445)
0812 19:01:23.489925 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:23.489933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:23.490862 (+ 929us) server_negotiation.cc:407] Connection header received
0812 19:01:23.490993 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:23.490999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:23.491200 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:23.491331 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:23.491811 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.493696 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.495182 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.495957 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.496105 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.496248 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:23.496759 (+ 511us) server_negotiation.cc:299] Negotiation successful
0812 19:01:23.496855 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":118,"threads_started":1}
I20260812 19:01:23.500356 13113 tablet_service.cc:1467] Processing CreateTablet for tablet cb487d2ed32d4ac898d991a0d14b0274 (DEFAULT_TABLE table=test-table [id=7d20f5e7e9254a77ba62934ecd60f492]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:23.501480 13113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb487d2ed32d4ac898d991a0d14b0274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:23.501464 12960 tablet_service.cc:1467] Processing CreateTablet for tablet cb487d2ed32d4ac898d991a0d14b0274 (DEFAULT_TABLE table=test-table [id=7d20f5e7e9254a77ba62934ecd60f492]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:23.502450 12960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb487d2ed32d4ac898d991a0d14b0274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:23.502882 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.489485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:45515 (local address 127.0.0.1:52102)
0812 19:01:23.493992 (+ 4507us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:23.494019 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:23.494107 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:23.498722 (+ 4615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:23.498725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:23.498742 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:23.498823 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:23.498829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.500188 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.500191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:23.501361 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:23.501372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.502469 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.502473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:23.502558 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.502583 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 19:01:23.502679 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 19:01:23.502754 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4443,"thread_start_us":86,"threads_started":1}
I20260812 19:01:23.502895 13387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.489935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52102 (local address 127.29.112.195:45515)
0812 19:01:23.498090 (+ 8155us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:23.498097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:23.498125 (+ 28us) server_negotiation.cc:407] Connection header received
0812 19:01:23.498202 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:23.498208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:23.498413 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:23.498566 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:23.498971 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.500067 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.501514 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.502333 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.502459 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.502596 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:23.502685 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 19:01:23.502770 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8024,"thread_start_us":100,"threads_started":1}
I20260812 19:01:23.505344 13272 tablet_service.cc:1467] Processing CreateTablet for tablet cb487d2ed32d4ac898d991a0d14b0274 (DEFAULT_TABLE table=test-table [id=7d20f5e7e9254a77ba62934ecd60f492]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:23.506315 13272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb487d2ed32d4ac898d991a0d14b0274. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:23.508641 13389 tablet_bootstrap.cc:492] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3: Bootstrap starting.
I20260812 19:01:23.510855 13390 tablet_bootstrap.cc:492] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: Bootstrap starting.
I20260812 19:01:23.511826 13389 tablet_bootstrap.cc:654] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:23.512795 13389 log.cc:824] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:23.512939 13390 tablet_bootstrap.cc:654] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:23.513720 13390 log.cc:824] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:23.514459 13392 tablet_bootstrap.cc:492] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463: Bootstrap starting.
I20260812 19:01:23.515240 13389 tablet_bootstrap.cc:492] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3: No bootstrap required, opened a new log
I20260812 19:01:23.515445 13389 ts_tablet_manager.cc:1397] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 19:01:23.516492 13392 tablet_bootstrap.cc:654] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:23.518363 13390 tablet_bootstrap.cc:492] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: No bootstrap required, opened a new log
I20260812 19:01:23.518553 13390 ts_tablet_manager.cc:1397] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 19:01:23.518726 13389 raft_consensus.cc:348] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39829 } } peers { permanent_uuid: "b0b83e9efa5443f5af45b1444715d0a3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38445 } } peers { permanent_uuid: "c7aff0658cb140b1abf27c64d3c80463" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45515 } }
I20260812 19:01:23.518955 13389 raft_consensus.cc:374] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:23.519044 13389 raft_consensus.cc:729] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b83e9efa5443f5af45b1444715d0a3, State: Initialized, Role: FOLLOWER
I20260812 19:01:23.519495 13389 consensus_queue.cc:260] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39829 } } peers { permanent_uuid: "b0b83e9efa5443f5af45b1444715d0a3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38445 } } peers { permanent_uuid: "c7aff0658cb140b1abf27c64d3c80463" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45515 } }
I20260812 19:01:23.520114 13033 heartbeater.cc:503] Master 127.29.112.254:36363 was elected leader, sending a full tablet report...
I20260812 19:01:23.520522 13389 ts_tablet_manager.cc:1428] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 19:01:23.521660 13392 log.cc:824] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:23.523164 13390 raft_consensus.cc:348] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39829 } } peers { permanent_uuid: "b0b83e9efa5443f5af45b1444715d0a3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38445 } } peers { permanent_uuid: "c7aff0658cb140b1abf27c64d3c80463" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45515 } }
I20260812 19:01:23.523581 13390 raft_consensus.cc:374] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:23.523739 13390 raft_consensus.cc:729] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c438ebd25782425c9e0f3ce42ccdae8c, State: Initialized, Role: FOLLOWER
I20260812 19:01:23.523972 13392 tablet_bootstrap.cc:492] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463: No bootstrap required, opened a new log
I20260812 19:01:23.524165 13392 ts_tablet_manager.cc:1397] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260812 19:01:23.524461 13390 consensus_queue.cc:260] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39829 } } peers { permanent_uuid: "b0b83e9efa5443f5af45b1444715d0a3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38445 } } peers { permanent_uuid: "c7aff0658cb140b1abf27c64d3c80463" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45515 } }
I20260812 19:01:23.525703 13390 ts_tablet_manager.cc:1428] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 19:01:23.526127 13190 heartbeater.cc:503] Master 127.29.112.254:36363 was elected leader, sending a full tablet report...
I20260812 19:01:23.528463 13392 raft_consensus.cc:348] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39829 } } peers { permanent_uuid: "b0b83e9efa5443f5af45b1444715d0a3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38445 } } peers { permanent_uuid: "c7aff0658cb140b1abf27c64d3c80463" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45515 } }
I20260812 19:01:23.528781 13392 raft_consensus.cc:374] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:23.528885 13392 raft_consensus.cc:729] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7aff0658cb140b1abf27c64d3c80463, State: Initialized, Role: FOLLOWER
I20260812 19:01:23.529444 13392 consensus_queue.cc:260] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39829 } } peers { permanent_uuid: "b0b83e9efa5443f5af45b1444715d0a3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38445 } } peers { permanent_uuid: "c7aff0658cb140b1abf27c64d3c80463" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45515 } }
I20260812 19:01:23.530395 13349 heartbeater.cc:503] Master 127.29.112.254:36363 was elected leader, sending a full tablet report...
I20260812 19:01:23.530632 13392 ts_tablet_manager.cc:1428] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260812 19:01:23.635279 13034 tablet.cc:2369] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 19:01:23 dist-test-slave-6nmv krb5kdc[11646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561283, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for HTTP/127.29.112.212@KRBTEST.COM
W20260812 19:01:23.675277 13191 tablet.cc:2369] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:01:23.705832 13350 tablet.cc:2369] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:01:23.793398 13399 raft_consensus.cc:482] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:01:23.793653 13399 raft_consensus.cc:504] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39829 } } peers { permanent_uuid: "b0b83e9efa5443f5af45b1444715d0a3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38445 } } peers { permanent_uuid: "c7aff0658cb140b1abf27c64d3c80463" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45515 } }
I20260812 19:01:23.795059 13399 leader_election.cc:288] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0b83e9efa5443f5af45b1444715d0a3 (127.29.112.193:38445), c7aff0658cb140b1abf27c64d3c80463 (127.29.112.195:45515)
I20260812 19:01:23.801560 13384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.796244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:51191 (local address 127.29.112.193:38445)
0812 19:01:23.796413 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:23.796418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:23.796867 (+ 449us) server_negotiation.cc:407] Connection header received
0812 19:01:23.797133 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:23.797137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:23.797191 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:23.797243 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:23.797702 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.798537 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.800188 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.800779 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.800920 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.800992 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:23.801342 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 19:01:23.801423 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 19:01:23.802237 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.795322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:38445 (local address 127.29.112.194:51191)
0812 19:01:23.796773 (+ 1451us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:23.796789 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:23.796992 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:23.797399 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:23.797401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:23.797420 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:23.797568 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:23.797575 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.798691 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.798695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:23.800039 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:23.800050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.801091 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.801094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:23.801190 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.801217 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 19:01:23.802014 (+ 797us) client_negotiation.cc:240] Negotiation successful
0812 19:01:23.802098 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1380,"thread_start_us":134,"threads_started":1}
I20260812 19:01:23.803119 12982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb487d2ed32d4ac898d991a0d14b0274" candidate_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b83e9efa5443f5af45b1444715d0a3" is_pre_election: true
I20260812 19:01:23.803437 12982 raft_consensus.cc:2454] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c438ebd25782425c9e0f3ce42ccdae8c in term 0.
I20260812 19:01:23.803974 13062 leader_election.cc:302] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0b83e9efa5443f5af45b1444715d0a3, c438ebd25782425c9e0f3ce42ccdae8c; no voters:
I20260812 19:01:23.804332 13399 raft_consensus.cc:2781] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:01:23.804421 13399 raft_consensus.cc:482] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:01:23.804468 13399 raft_consensus.cc:3037] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:23.805850 13399 raft_consensus.cc:504] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39829 } } peers { permanent_uuid: "b0b83e9efa5443f5af45b1444715d0a3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38445 } } peers { permanent_uuid: "c7aff0658cb140b1abf27c64d3c80463" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45515 } }
I20260812 19:01:23.806277 13399 leader_election.cc:288] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [CANDIDATE]: Term 1 election: Requested vote from peers b0b83e9efa5443f5af45b1444715d0a3 (127.29.112.193:38445), c7aff0658cb140b1abf27c64d3c80463 (127.29.112.195:45515)
I20260812 19:01:23.806910 12982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb487d2ed32d4ac898d991a0d14b0274" candidate_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b83e9efa5443f5af45b1444715d0a3"
I20260812 19:01:23.807050 12982 raft_consensus.cc:3037] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:23.808460 12982 raft_consensus.cc:2454] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c438ebd25782425c9e0f3ce42ccdae8c in term 1.
I20260812 19:01:23.808895 13062 leader_election.cc:302] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0b83e9efa5443f5af45b1444715d0a3, c438ebd25782425c9e0f3ce42ccdae8c; no voters:
I20260812 19:01:23.809127 13399 raft_consensus.cc:2781] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:01:23.809381 13399 raft_consensus.cc:686] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 1 LEADER]: Becoming Leader. State: Replica: c438ebd25782425c9e0f3ce42ccdae8c, State: Running, Role: LEADER
I20260812 19:01:23.809773 13399 consensus_queue.cc:237] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39829 } } peers { permanent_uuid: "b0b83e9efa5443f5af45b1444715d0a3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38445 } } peers { permanent_uuid: "c7aff0658cb140b1abf27c64d3c80463" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45515 } }
I20260812 19:01:23.813624 12812 catalog_manager.cc:5495] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c reported cstate change: term changed from 0 to 1, leader changed from <none> to c438ebd25782425c9e0f3ce42ccdae8c (127.29.112.194). New cstate: current_term: 1 leader_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 39829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b83e9efa5443f5af45b1444715d0a3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aff0658cb140b1abf27c64d3c80463" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45515 } health_report { overall_health: UNKNOWN } } }
I20260812 19:01:23.819480 13387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.796404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:49409 (local address 127.29.112.195:45515)
0812 19:01:23.796526 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:23.796530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:23.802982 (+ 6452us) server_negotiation.cc:407] Connection header received
0812 19:01:23.803173 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:23.803177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:23.803229 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:23.803280 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:23.815934 (+ 12654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.816664 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.818135 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.818738 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.818840 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.818909 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:23.819249 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 19:01:23.819337 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 19:01:23.820065 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.795745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:45515 (local address 127.29.112.194:49409)
0812 19:01:23.802900 (+ 7155us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:23.802915 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:23.803054 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:23.803490 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:23.803493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:23.803584 (+ 91us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:23.815564 (+ 11980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:23.815579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.816828 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.816834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:23.817960 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:23.817973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.819010 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.819014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:23.819104 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.819128 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 19:01:23.819829 (+ 701us) client_negotiation.cc:240] Negotiation successful
0812 19:01:23.819926 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7071,"thread_start_us":92,"threads_started":1}
I20260812 19:01:23.821041 13297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb487d2ed32d4ac898d991a0d14b0274" candidate_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7aff0658cb140b1abf27c64d3c80463" is_pre_election: true
I20260812 19:01:23.821378 13297 raft_consensus.cc:2454] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c438ebd25782425c9e0f3ce42ccdae8c in term 0.
I20260812 19:01:23.821725 13296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb487d2ed32d4ac898d991a0d14b0274" candidate_uuid: "c438ebd25782425c9e0f3ce42ccdae8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7aff0658cb140b1abf27c64d3c80463"
I20260812 19:01:23.821854 13296 raft_consensus.cc:3037] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:23.823405 13296 raft_consensus.cc:2454] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c438ebd25782425c9e0f3ce42ccdae8c in term 1.
I20260812 19:01:23.867484 13388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.862433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40006 (local address 127.29.112.194:39829)
0812 19:01:23.862596 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:23.862601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:23.862624 (+ 23us) server_negotiation.cc:407] Connection header received
0812 19:01:23.862741 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:23.862745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:23.862811 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:23.862919 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:01:23.863274 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.864034 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.865287 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:23.865584 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:23.865690 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:23.866147 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:01:23.866714 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:01:23.866826 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:23.867277 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 19:01:23.867339 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 19:01:23.877935 13296 raft_consensus.cc:1261] T cb487d2ed32d4ac898d991a0d14b0274 P c7aff0658cb140b1abf27c64d3c80463 [term 1 FOLLOWER]: Refusing update from remote peer c438ebd25782425c9e0f3ce42ccdae8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:01:23.878813 13405 consensus_queue.cc:1035] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aff0658cb140b1abf27c64d3c80463" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 45515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:01:23.886354 12982 raft_consensus.cc:1261] T cb487d2ed32d4ac898d991a0d14b0274 P b0b83e9efa5443f5af45b1444715d0a3 [term 1 FOLLOWER]: Refusing update from remote peer c438ebd25782425c9e0f3ce42ccdae8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:01:23.887279 13405 consensus_queue.cc:1035] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b83e9efa5443f5af45b1444715d0a3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 38445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:01:23.904856 13408 mvcc.cc:205] Tried to move back new op lower bound from 7317755018754957312 to 7317755018489081856. Current Snapshot: MvccSnapshot[applied={T|T < 7317755018754957312}]
I20260812 19:01:23.906651 13412 mvcc.cc:205] Tried to move back new op lower bound from 7317755018754957312 to 7317755018489081856. Current Snapshot: MvccSnapshot[applied={T|T < 7317755018754957312}]
I20260812 19:01:23.911844 13409 mvcc.cc:205] Tried to move back new op lower bound from 7317755018754957312 to 7317755018489081856. Current Snapshot: MvccSnapshot[applied={T|T < 7317755018754957312}]
I20260812 19:01:23.922690 12812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:01:23.922864 12812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:01:23.925565 12812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9459741414841b5ac78e81e03d7a5d3: Sending DeleteTablet for 3 replicas of tablet cb487d2ed32d4ac898d991a0d14b0274
I20260812 19:01:23.926446 13113 tablet_service.cc:1514] Processing DeleteTablet for tablet cb487d2ed32d4ac898d991a0d14b0274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:23 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:39998
I20260812 19:01:23.927588 12960 tablet_service.cc:1514] Processing DeleteTablet for tablet cb487d2ed32d4ac898d991a0d14b0274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:23 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:48500
I20260812 19:01:23.927620 13272 tablet_service.cc:1514] Processing DeleteTablet for tablet cb487d2ed32d4ac898d991a0d14b0274 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:23 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:52102
I20260812 19:01:23.928927 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 12854
I20260812 19:01:23.940582 13418 tablet_replica.cc:330] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: stopping tablet replica
I20260812 19:01:23.941064 13418 raft_consensus.cc:2229] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:01:23.941653 13418 raft_consensus.cc:2258] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:01:23.953861 13418 ts_tablet_manager.cc:1905] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:01:23.957443 13418 ts_tablet_manager.cc:1918] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:01:23.957557 13418 log.cc:1196] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/cb487d2ed32d4ac898d991a0d14b0274
I20260812 19:01:23.957931 13418 ts_tablet_manager.cc:1939] T cb487d2ed32d4ac898d991a0d14b0274 P c438ebd25782425c9e0f3ce42ccdae8c: Deleting consensus metadata
I20260812 19:01:23.959277 12788 catalog_manager.cc:4841] TS c438ebd25782425c9e0f3ce42ccdae8c (127.29.112.194:39829): tablet cb487d2ed32d4ac898d991a0d14b0274 (table test-table [id=7d20f5e7e9254a77ba62934ecd60f492]) successfully deleted
W20260812 19:01:23.962630 12790 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:38445 (error 108)
I20260812 19:01:23.964221 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 13037
I20260812 19:01:23.987416 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 13194
I20260812 19:01:23.987515 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:23.987102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:38445 (local address 127.0.0.1:48516)
0812 19:01:23.987301 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:23.987405 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:38445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 19:01:23.989718 12790 catalog_manager.cc:4584] TS b0b83e9efa5443f5af45b1444715d0a3 (127.29.112.193:38445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb487d2ed32d4ac898d991a0d14b0274: Network error: Client connection negotiation failed: client connection to 127.29.112.193:38445: connect: Connection refused (error 111)
I20260812 19:01:24.003715 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:24.003319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:45515 (local address 127.0.0.1:52104)
0812 19:01:24.003545 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:24.003606 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.195:45515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260812 19:01:24.004211 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 12768
I20260812 19:01:24.021381 30147 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 19:01:24.366346 30147 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 19:01:24.366701 30147 mini_ranger.cc:66] Stopping Ranger...
I20260812 19:01:24.710996 30147 mini_ranger.cc:68] Stopped Ranger
2026-08-12 19:01:24.711 UTC [11687] LOG: received smart shutdown request
2026-08-12 19:01:24.715 UTC [11687] LOG: background worker "logical replication launcher" (PID 11698) exited with exit code 1
2026-08-12 19:01:24.715 UTC [11691] LOG: shutting down
2026-08-12 19:01:24.855 UTC [11687] LOG: database system is shut down
2026-08-12T19:01:24Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47732 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:01:25 dist-test-slave-6nmv krb5kdc[13439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:01:25 dist-test-slave-6nmv krb5kdc[13439](info): set up 2 sockets
Aug 12 19:01:25 dist-test-slave-6nmv krb5kdc[13439](info): commencing operation
krb5kdc: starting...
W20260812 19:01:26.073998 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:01:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:01:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:01:26.192134 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:44841
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:44841,127.29.112.253:46609,127.29.112.252:35605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:26.298031 13457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:26.298290 13457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:26.298377 13457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:26.302147 13457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:01:26.302207 13457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:26.302240 13457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:26.302291 13457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:01:26.302327 13457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:01:26.307005 13457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:44841,127.29.112.253:46609,127.29.112.252:35605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:44841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:26.307974 13457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:26.308933 13457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:26.316907 13462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:26.317023 13463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:26.317061 13465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:26.317093 13457 server_base.cc:1034] running on GCE node
I20260812 19:01:26.317859 13457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:26.318886 13457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:26.320075 13457 hybrid_clock.cc:648] HybridClock initialized: now 1786561286320062 us; error 25 us; skew 500 ppm
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:26.322716 13457 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:01:26.323582 13457 webserver.cc:460] Webserver started at http://127.29.112.254:39889/ using document root <none> and password file <none>
I20260812 19:01:26.324107 13457 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:26.324193 13457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:26.324415 13457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:26.326316 13457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "8cc582ee72be41ad863cb9ea4d95691b"
format_stamp: "Formatted at 2026-08-12 19:01:26 on dist-test-slave-6nmv"
server_key: "88c4afd8cfb3c33fdc17deb798a11b56"
server_key_iv: "b1ad04b3e3a3e7a933f15b9280456281"
server_key_version: "encryptionkey@0"
I20260812 19:01:26.326848 13457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "8cc582ee72be41ad863cb9ea4d95691b"
format_stamp: "Formatted at 2026-08-12 19:01:26 on dist-test-slave-6nmv"
server_key: "88c4afd8cfb3c33fdc17deb798a11b56"
server_key_iv: "b1ad04b3e3a3e7a933f15b9280456281"
server_key_version: "encryptionkey@0"
I20260812 19:01:26.330365 13457 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:01:26.332617 13473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:26.333649 13457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:01:26.333776 13457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "8cc582ee72be41ad863cb9ea4d95691b"
format_stamp: "Formatted at 2026-08-12 19:01:26 on dist-test-slave-6nmv"
server_key: "88c4afd8cfb3c33fdc17deb798a11b56"
server_key_iv: "b1ad04b3e3a3e7a933f15b9280456281"
server_key_version: "encryptionkey@0"
I20260812 19:01:26.333915 13457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:26.343701 13457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:26.346565 13457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:26.346736 13457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:26.353423 13457 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:44841
I20260812 19:01:26.353690 13529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:44841 every 8 connection(s)
I20260812 19:01:26.354269 13457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:01:26.356781 13530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:26.357753 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 13457
I20260812 19:01:26.357916 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:01:26.358311 30147 external_mini_cluster.cc:1428] Setting key a2ee85f2e599e915f63df49db28b317c
I20260812 19:01:26.361671 13530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.112.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.253@KRBTEST.COM" created.
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:26.376173 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.363917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50024 (local address 127.29.112.254:44841)
0812 19:01:26.364446 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.364458 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.364549 (+ 91us) server_negotiation.cc:407] Connection header received
0812 19:01:26.365496 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.365513 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.365734 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.365944 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.366674 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.367824 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.368832 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.369192 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.369310 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.371800 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.371808 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.371822 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.371852 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.373428 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.374040 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.374043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.374044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.374135 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.374472 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.374474 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.374475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.374774 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.374863 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.375281 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.375481 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":400,"thread_start_us":139,"threads_started":1}
I20260812 19:01:26.376178 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.363916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44841 (local address 127.0.0.1:50024)
0812 19:01:26.364375 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.364427 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:26.365191 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:26.365947 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:26.365951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:26.366244 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:26.366518 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.366531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.367980 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.367987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.368721 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.368730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.369320 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.369323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.369512 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.370085 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:26.370106 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:26.371638 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:26.373563 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.373574 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.373584 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.373938 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.374225 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.374231 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.374235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.374380 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.374879 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:26.374887 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:26.375047 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 19:01:26.375232 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 19:01:26.375578 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":70,"thread_start_us":135,"threads_started":1}
I20260812 19:01:26.378443 13530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:01:26.379106 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.378851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:46609 (local address 127.0.0.1:58394)
0812 19:01:26.378970 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.379056 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 19:01:26.379278 13474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.253:46609: connect: Connection refused (error 111)
I20260812 19:01:26.380028 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.379843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:46609 (local address 127.0.0.1:58406)
0812 19:01:26.379934 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.379984 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 19:01:26.380148 13530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.253:46609: Network error: Client connection negotiation failed: client connection to 127.29.112.253:46609: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab.
Entry for principal kudu/127.29.112.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab.
I20260812 19:01:26.384732 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.253:46609
--webserver_interface=127.29.112.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab
--principal=kudu/127.29.112.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:44841,127.29.112.253:46609,127.29.112.252:35605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:01:26.429385 13530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } attempt: 1
I20260812 19:01:26.430346 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.430017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:46609 (local address 127.0.0.1:58412)
0812 19:01:26.430191 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.430260 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 19:01:26.431026 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.430810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:46609 (local address 127.0.0.1:58422)
0812 19:01:26.430906 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.430966 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 19:01:26.431146 13530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.253:46609: Network error: Client connection negotiation failed: client connection to 127.29.112.253:46609: connect: Connection refused (error 111)
W20260812 19:01:26.490800 13537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:26.491066 13537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:26.491139 13537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:26.495555 13537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:01:26.495610 13537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:26.495628 13537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:26.495669 13537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:01:26.495698 13537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:01:26.499293 13530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } attempt: 2
I20260812 19:01:26.500120 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.499807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:46609 (local address 127.0.0.1:58428)
0812 19:01:26.499972 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.500037 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 19:01:26.500741 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.500551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:46609 (local address 127.0.0.1:58430)
0812 19:01:26.500639 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.500684 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:46609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 19:01:26.500856 13530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.253:46609: Network error: Client connection negotiation failed: client connection to 127.29.112.253:46609: connect: Connection refused (error 111)
I20260812 19:01:26.501482 13537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:44841,127.29.112.253:46609,127.29.112.252:35605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab
--principal=kudu/127.29.112.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.253:46609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.112.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:26.502519 13537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:26.503461 13537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:26.511826 13543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:26.511822 13546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:26.511831 13542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:26.512101 13537 server_base.cc:1034] running on GCE node
I20260812 19:01:26.512768 13537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:26.513851 13537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:26.515015 13537 hybrid_clock.cc:648] HybridClock initialized: now 1786561286515001 us; error 25 us; skew 500 ppm
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:26.517784 13537 init.cc:377] Logged in from keytab as kudu/127.29.112.253@KRBTEST.COM (short username kudu)
I20260812 19:01:26.518663 13537 webserver.cc:460] Webserver started at http://127.29.112.253:36393/ using document root <none> and password file <none>
I20260812 19:01:26.519188 13537 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:26.519248 13537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:26.519482 13537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:26.521397 13537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/data/instance:
uuid: "eb7a38b3de8e4cc085622be4cbf29301"
format_stamp: "Formatted at 2026-08-12 19:01:26 on dist-test-slave-6nmv"
server_key: "3383b901ac2be8aa9258ac3a9ec824e4"
server_key_iv: "b7efa97db945af12428247212a9ea150"
server_key_version: "encryptionkey@0"
I20260812 19:01:26.521968 13537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/wal/instance:
uuid: "eb7a38b3de8e4cc085622be4cbf29301"
format_stamp: "Formatted at 2026-08-12 19:01:26 on dist-test-slave-6nmv"
server_key: "3383b901ac2be8aa9258ac3a9ec824e4"
server_key_iv: "b7efa97db945af12428247212a9ea150"
server_key_version: "encryptionkey@0"
I20260812 19:01:26.525126 13537 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:01:26.527258 13554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:26.528384 13537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:01:26.528509 13537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/wal
uuid: "eb7a38b3de8e4cc085622be4cbf29301"
format_stamp: "Formatted at 2026-08-12 19:01:26 on dist-test-slave-6nmv"
server_key: "3383b901ac2be8aa9258ac3a9ec824e4"
server_key_iv: "b7efa97db945af12428247212a9ea150"
server_key_version: "encryptionkey@0"
I20260812 19:01:26.528602 13537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:26.546653 13537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:26.549623 13537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:26.549755 13537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:26.556303 13537 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.253:46609
I20260812 19:01:26.556558 13616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.253:46609 every 8 connection(s)
I20260812 19:01:26.557188 13537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
I20260812 19:01:26.559792 13617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:26.560281 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 13537
I20260812 19:01:26.560390 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-1/wal/instance
I20260812 19:01:26.560665 30147 external_mini_cluster.cc:1428] Setting key 19a9932b8601c280b8728610b4e20ece
I20260812 19:01:26.563251 13617 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.112.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.252@KRBTEST.COM" created.
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:26.576921 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.564886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.29.112.254:44841)
0812 19:01:26.565026 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.565031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.565727 (+ 696us) server_negotiation.cc:407] Connection header received
0812 19:01:26.566557 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.566562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.566632 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.566761 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.567807 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.568665 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.569800 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.570082 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.570172 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.573261 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.573264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.573266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.573289 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.574787 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.575273 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.575279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.575283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.575375 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.575722 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.575728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.575732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.576008 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.576113 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.576570 (+ 457us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.576708 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:01:26.577720 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.565233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44841 (local address 127.0.0.1:50028)
0812 19:01:26.565600 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.565636 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:26.566336 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:26.566880 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:26.566885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:26.567187 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:26.567630 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.567643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.568787 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.568790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.569680 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.569689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.570226 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.570229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.570440 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.571016 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:26.571031 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:26.573090 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:26.574895 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.574901 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.574918 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.575172 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.575500 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.575510 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.575511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.575635 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.576115 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:26.576119 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:26.576329 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 19:01:26.576471 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 19:01:26.576704 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":91,"threads_started":1}
I20260812 19:01:26.579016 13617 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab.
Entry for principal kudu/127.29.112.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab.
I20260812 19:01:26.587025 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.252:35605
--webserver_interface=127.29.112.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab
--principal=kudu/127.29.112.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:38351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:44841,127.29.112.253:46609,127.29.112.252:35605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:01:26.588114 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.579470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:46609 (local address 127.0.0.1:58432)
0812 19:01:26.579597 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.579611 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:26.579680 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:26.580344 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:26.580347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:26.580505 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:26.580579 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.580584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.581484 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.581487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.582085 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.582093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.582889 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.582892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.583009 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.583395 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:26.583434 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:26.584574 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:26.586618 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.586622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.586624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.586787 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.587069 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.587074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.587076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.587176 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.587623 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:26.587626 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:26.587710 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 19:01:26.587807 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 19:01:26.587928 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":2}
I20260812 19:01:26.588160 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.579782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58432 (local address 127.29.112.253:46609)
0812 19:01:26.580060 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.580068 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.580085 (+ 17us) server_negotiation.cc:407] Connection header received
0812 19:01:26.580114 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.580117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.580241 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.580364 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.580673 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.581380 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.582186 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.582744 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.582891 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.584695 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.584704 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.584720 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.584768 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.586525 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.586883 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.586887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.586889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.586965 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.587282 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.587285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.587287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.587511 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.587644 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.587896 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.588011 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":126,"threads_started":1}
I20260812 19:01:26.589956 13617 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:01:26.590692 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.590398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36688)
0812 19:01:26.590523 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.590609 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 19:01:26.590889 13556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
I20260812 19:01:26.591710 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.591429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36690)
0812 19:01:26.591546 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.591607 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 19:01:26.591878 13617 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:35605: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
I20260812 19:01:26.591991 13530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } attempt: 3
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
I20260812 19:01:26.600622 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.592407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:46609 (local address 127.0.0.1:58438)
0812 19:01:26.592617 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.592628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:26.592699 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:26.592933 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:26.592935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:26.593059 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:26.593141 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.593145 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.594163 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.594165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.594677 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.594684 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.595282 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.595284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.595371 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.595811 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:26.595823 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:26.597422 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:26.599179 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.599181 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.599182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.599365 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.599654 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.599656 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.599657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.599724 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.600152 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:26.600156 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:26.600244 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 19:01:26.600339 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 19:01:26.600472 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260812 19:01:26.600981 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.592426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58438 (local address 127.29.112.253:46609)
0812 19:01:26.592532 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.592536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.592693 (+ 157us) server_negotiation.cc:407] Connection header received
0812 19:01:26.592761 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.592766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.592825 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.592934 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.593239 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.594059 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.594784 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.595178 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.595298 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.597543 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.597547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.597549 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.597572 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.599080 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.599454 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.599457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.599459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.599552 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.599801 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.599804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.599805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.600008 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.600107 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.600463 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.600723 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 19:01:26.611982 13530 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:01:26.612843 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.612525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36694)
0812 19:01:26.612701 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.612753 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 19:01:26.613617 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.613347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36708)
0812 19:01:26.613501 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.613554 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260812 19:01:26.613730 13530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:35605: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
I20260812 19:01:26.641211 13617 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } attempt: 1
I20260812 19:01:26.642102 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.641753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36722)
0812 19:01:26.641932 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.642002 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 19:01:26.642777 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.642572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36726)
0812 19:01:26.642669 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.642719 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 19:01:26.642931 13617 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:35605: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
I20260812 19:01:26.644893 13530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } attempt: 1
I20260812 19:01:26.645625 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.645331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36742)
0812 19:01:26.645487 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.645549 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 19:01:26.646423 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.646176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36752)
0812 19:01:26.646293 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.646345 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 19:01:26.646617 13530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:35605: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
W20260812 19:01:26.703035 13623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:26.703323 13623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:26.703387 13623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:26.707120 13623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:01:26.707194 13623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:26.707231 13623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:26.707280 13623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:01:26.707316 13623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:01:26.711133 13617 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } attempt: 2
I20260812 19:01:26.711990 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.711640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36754)
0812 19:01:26.711827 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.711898 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 19:01:26.712674 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.712466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36756)
0812 19:01:26.712562 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.712615 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 19:01:26.712785 13623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:38351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:44841,127.29.112.253:46609,127.29.112.252:35605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab
--principal=kudu/127.29.112.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.252:35605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.112.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
W20260812 19:01:26.712826 13617 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:35605: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
I20260812 19:01:26.713948 13623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:26.714910 13623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:01:26.721791 13530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } attempt: 2
I20260812 19:01:26.722652 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.722300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36770)
0812 19:01:26.722491 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.722552 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 19:01:26.722620 13634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:26.722781 13631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:26.722947 13630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:26.723313 13623 server_base.cc:1034] running on GCE node
I20260812 19:01:26.723558 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.723245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36784)
0812 19:01:26.723449 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.723496 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260812 19:01:26.723709 13530 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:35605: Network error: Client connection negotiation failed: client connection to 127.29.112.252:35605: connect: Connection refused (error 111)
I20260812 19:01:26.723871 13623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:26.724823 13623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:26.725972 13623 hybrid_clock.cc:648] HybridClock initialized: now 1786561286725966 us; error 33 us; skew 500 ppm
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:26.728597 13623 init.cc:377] Logged in from keytab as kudu/127.29.112.252@KRBTEST.COM (short username kudu)
I20260812 19:01:26.729416 13623 webserver.cc:460] Webserver started at http://127.29.112.252:34759/ using document root <none> and password file <none>
I20260812 19:01:26.730016 13623 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:26.730079 13623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:26.730293 13623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:26.732300 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/data/instance:
uuid: "c046156e81444f6f92e0c1bd539c3b71"
format_stamp: "Formatted at 2026-08-12 19:01:26 on dist-test-slave-6nmv"
server_key: "d7799d39a4b687ee2bba0bc8f68b21a8"
server_key_iv: "2eea7a994e2f9b099bacbd20cdaa6042"
server_key_version: "encryptionkey@0"
I20260812 19:01:26.732887 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/wal/instance:
uuid: "c046156e81444f6f92e0c1bd539c3b71"
format_stamp: "Formatted at 2026-08-12 19:01:26 on dist-test-slave-6nmv"
server_key: "d7799d39a4b687ee2bba0bc8f68b21a8"
server_key_iv: "2eea7a994e2f9b099bacbd20cdaa6042"
server_key_version: "encryptionkey@0"
I20260812 19:01:26.736393 13623 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 19:01:26.738790 13641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:26.740026 13623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:01:26.740152 13623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/wal
uuid: "c046156e81444f6f92e0c1bd539c3b71"
format_stamp: "Formatted at 2026-08-12 19:01:26 on dist-test-slave-6nmv"
server_key: "d7799d39a4b687ee2bba0bc8f68b21a8"
server_key_iv: "2eea7a994e2f9b099bacbd20cdaa6042"
server_key_version: "encryptionkey@0"
I20260812 19:01:26.740247 13623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:26.750330 13623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:26.752893 13623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:26.753072 13623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:26.759459 13623 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.252:35605
I20260812 19:01:26.759747 13700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.252:35605 every 8 connection(s)
I20260812 19:01:26.760376 13623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
I20260812 19:01:26.762985 13701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:26.764055 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 13623
I20260812 19:01:26.764168 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786561125017040-30147-0/minicluster-data/master-2/wal/instance
I20260812 19:01:26.764451 30147 external_mini_cluster.cc:1428] Setting key fd53b7138e9cadc4019021e2dca10b82
I20260812 19:01:26.767246 13701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:26.778112 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.769063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50036 (local address 127.29.112.254:44841)
0812 19:01:26.769191 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.769195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.769432 (+ 237us) server_negotiation.cc:407] Connection header received
0812 19:01:26.769590 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.769593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.769647 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.769710 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.770296 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.770816 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.771744 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.772112 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.772234 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.774726 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.774728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.774730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.774751 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.776320 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.776769 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.776772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.776774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.776861 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.777175 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.777178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.777179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.777420 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.777523 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.777852 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.777965 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:01:26.778460 13705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.768914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44841 (local address 127.0.0.1:50036)
0812 19:01:26.769308 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.769351 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:26.769471 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:26.769773 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:26.769781 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:26.770083 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:26.770186 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.770191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.770920 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.770925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.771607 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.771625 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.772257 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.772259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.772333 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.772881 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:26.772905 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:26.774589 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:26.776441 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.776468 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.776472 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.776675 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.776948 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.776951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.776953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.777054 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.777542 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:26.777547 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:26.777638 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 19:01:26.777748 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 19:01:26.777919 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":139,"threads_started":1}
I20260812 19:01:26.778460 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.765653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.29.112.252:35605)
0812 19:01:26.766047 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.766064 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.766105 (+ 41us) server_negotiation.cc:407] Connection header received
0812 19:01:26.766885 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.766908 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.767289 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.767627 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.768300 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.769668 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.770542 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.771013 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.771149 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.774026 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.774033 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.774045 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.774072 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.775602 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.775989 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.775995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.776001 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.776129 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.776396 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.776402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.776407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.776810 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.776996 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.777314 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.777599 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":142,"threads_started":1}
I20260812 19:01:26.779668 13701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:26.787691 13705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.780093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:46609 (local address 127.0.0.1:58452)
0812 19:01:26.780261 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.780275 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:26.780347 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:26.780581 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:26.780583 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:26.780751 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:26.780860 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.780869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.781754 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.781757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.782328 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.782337 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.782797 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.782801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.782909 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.783274 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:26.783292 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:26.784483 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:26.786326 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.786329 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.786330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.786504 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.786760 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.786764 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.786766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.786842 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.787231 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:26.787233 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:26.787327 (+ 94us) client_negotiation.cc:769] Sending connection context
0812 19:01:26.787424 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 19:01:26.787544 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260812 19:01:26.787748 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.780219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.29.112.253:46609)
0812 19:01:26.780358 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.780363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.780381 (+ 18us) server_negotiation.cc:407] Connection header received
0812 19:01:26.780425 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.780428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.780475 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.780566 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.780987 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.781645 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.782452 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.782689 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.782764 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.784612 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.784616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.784618 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.784650 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.786232 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.786599 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.786602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.786604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.786680 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.786926 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.786930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.786931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.787140 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.787238 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.787487 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.787603 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:01:26.788491 13701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:01:26.789939 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.780797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50040 (local address 127.29.112.254:44841)
0812 19:01:26.781115 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.781118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.781129 (+ 11us) server_negotiation.cc:407] Connection header received
0812 19:01:26.781158 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.781161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.781195 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.781268 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.781899 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.782352 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.783340 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.783531 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.783605 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.786205 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.786207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.786209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.786229 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.787671 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.788567 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.788570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.788571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.788665 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.789071 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.789074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.789077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.789253 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.789408 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.789632 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.789742 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
I20260812 19:01:26.797160 13705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.788908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36798)
0812 19:01:26.789480 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.789491 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:26.789592 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:26.789773 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:26.789775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:26.789953 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:26.790050 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.790058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.791324 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.791327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.791909 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.791917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.792361 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.792364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.792441 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.792873 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:26.792885 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:26.794182 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:26.795818 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.795823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.795826 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.796010 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.796247 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.796251 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.796253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.796322 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.796706 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:26.796709 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:26.796780 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 19:01:26.796869 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 19:01:26.796989 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518}
I20260812 19:01:26.797170 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.788991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36798 (local address 127.29.112.252:35605)
0812 19:01:26.789081 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.789085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.789559 (+ 474us) server_negotiation.cc:407] Connection header received
0812 19:01:26.789656 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.789661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.789691 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.789766 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.790160 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.791229 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.792033 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.792266 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.792339 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.794320 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.794322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.794324 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.794343 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.795722 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.796095 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.796100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.796102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.796168 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.796397 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.796402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.796403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.796603 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.796688 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.796918 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.797048 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":31}
I20260812 19:01:26.799094 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.791112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.29.112.253:46609)
0812 19:01:26.791211 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.791214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.791227 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:01:26.791336 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.791339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.791380 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.791453 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.792030 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.792597 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.793393 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.793610 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.793689 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.795868 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.795870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.795872 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.795890 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.797163 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.797925 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.797927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.797928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.798025 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.798269 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.798272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.798274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.798520 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.798757 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.798827 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.798935 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 19:01:26.800307 13701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71: Bootstrap starting.
I20260812 19:01:26.802187 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:01:26.803110 13701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:26.803995 13617 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } attempt: 3
I20260812 19:01:26.804034 13701 log.cc:824] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:26.807562 13701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71: No bootstrap required, opened a new log
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
I20260812 19:01:26.811820 13701 raft_consensus.cc:348] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } }
I20260812 19:01:26.812081 13701 raft_consensus.cc:374] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:26.812145 13701 raft_consensus.cc:729] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c046156e81444f6f92e0c1bd539c3b71, State: Initialized, Role: FOLLOWER
I20260812 19:01:26.812695 13701 consensus_queue.cc:260] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } }
I20260812 19:01:26.813941 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.804360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36808)
0812 19:01:26.804657 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.804672 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:26.804745 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:26.805241 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:26.805246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:26.805414 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:26.805522 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.805529 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.806540 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.806544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.807198 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.807210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.807802 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.807806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.807894 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.808811 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:26.808825 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:26.810259 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:26.812321 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.812325 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.812327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.812565 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.812912 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.812915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.812917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.813032 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.813469 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:26.813472 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:26.813559 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 19:01:26.813640 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 19:01:26.813763 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199}
I20260812 19:01:26.814093 13701 sys_catalog.cc:564] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:01:26.814317 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.804450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.29.112.254:44841)
0812 19:01:26.804609 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.804616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.805157 (+ 541us) server_negotiation.cc:407] Connection header received
0812 19:01:26.805326 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.805330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.805369 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.805417 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.806447 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.807104 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.808251 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.808577 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.808682 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.810332 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.810336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.810339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.810364 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.812063 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.812708 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.812713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.812715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.812799 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.813236 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.813240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.813243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.813486 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.813595 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.814028 (+ 433us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.814154 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 19:01:26.814538 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.804597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36808 (local address 127.29.112.252:35605)
0812 19:01:26.805030 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.805038 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.805051 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:01:26.805085 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.805089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.805138 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.805224 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.805651 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.806362 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.807322 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.807679 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.808093 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.810395 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.810399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.810401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.810422 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.812199 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.812702 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.812708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.812713 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.812796 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.813147 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.813149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.813151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.813357 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.814216 (+ 859us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.814286 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.814396 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372}
I20260812 19:01:26.817145 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.805137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58478 (local address 127.29.112.253:46609)
0812 19:01:26.805229 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.805233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.806709 (+ 1476us) server_negotiation.cc:407] Connection header received
0812 19:01:26.806841 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.806844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.806877 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.806917 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.807778 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.808596 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.810057 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.810354 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.810422 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.812936 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.812940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.812942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.812970 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.814686 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.815140 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.815143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.815145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.815226 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.815559 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.815563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.815566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.815774 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.815883 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.816918 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.817019 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 19:01:26.817505 13617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301: Bootstrap starting.
W20260812 19:01:26.819377 13731 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:01:26.819510 13731 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:01:26.820924 13617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:26.821878 13617 log.cc:824] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:26.822407 13718 sys_catalog.cc:455] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } } }
I20260812 19:01:26.822561 13718 sys_catalog.cc:458] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:01:26.822883 13530 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } attempt: 3
I20260812 19:01:26.823912 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.804778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36820 (local address 127.29.112.252:35605)
0812 19:01:26.805314 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.805319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.805330 (+ 11us) server_negotiation.cc:407] Connection header received
0812 19:01:26.805360 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.805364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.805402 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.805483 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.816440 (+ 10957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.817105 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.818412 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.818711 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.818790 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.820378 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.820382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.820384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.820406 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.822003 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.822513 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.822517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.822519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.822609 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.822945 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.822948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.822950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.823163 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.823258 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.823652 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.823759 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":118,"threads_started":1}
I20260812 19:01:26.819761 13623 master_runner.cc:186] Error getting master registration for 127.29.112.254:44841: OK, instance_id { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" instance_seqno: 1786561286351622 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 19:01:26.824298 13623 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:01:26.826236 13617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301: No bootstrap required, opened a new log
Aug 12 19:01:26 dist-test-slave-6nmv krb5kdc[13439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561286, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
I20260812 19:01:26.830310 13617 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } }
I20260812 19:01:26.830544 13617 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:26.830603 13617 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7a38b3de8e4cc085622be4cbf29301, State: Initialized, Role: FOLLOWER
I20260812 19:01:26.831073 13617 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } }
I20260812 19:01:26.831576 13734 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } } }
I20260812 19:01:26.831746 13734 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:01:26.832433 13617 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:01:26.833176 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.823245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:35605 (local address 127.0.0.1:36830)
0812 19:01:26.823362 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:26.823378 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:26.823455 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:26.823843 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:26.823846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:26.824036 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:26.824155 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.824162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.825091 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.825097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.825902 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:26.825912 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.826510 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.826513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:26.826571 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.827002 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:26.827020 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:26.828212 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:26.831500 (+ 3288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.831503 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.831505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.831785 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.832133 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:26.832137 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:26.832139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:26.832224 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:26.832655 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:26.832659 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:26.832755 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 19:01:26.832878 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 19:01:26.833020 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 19:01:26.833248 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:26.823473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.29.112.252:35605)
0812 19:01:26.823591 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:26.823596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:26.823607 (+ 11us) server_negotiation.cc:407] Connection header received
0812 19:01:26.823635 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:26.823638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:26.823676 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:26.823745 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:26.824289 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.824952 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.826022 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:26.826295 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:26.826405 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:26.828434 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:26.828438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:26.828440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:26.828463 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:26.831363 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.831938 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.831943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.831946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.832040 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:26.832360 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:26.832364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:26.832365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:26.832571 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:26.832662 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:26.832967 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 19:01:26.833115 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 19:01:26.836591 13530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b: Bootstrap starting.
W20260812 19:01:26.838533 13747 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:01:26.838604 13747 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:01:26.838680 13537 master_runner.cc:186] Error getting master registration for 127.29.112.254:44841: OK, instance_id { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" instance_seqno: 1786561286351622 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 19:01:26.840874 13537 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:01:26.842487 13530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:26.843159 13530 log.cc:824] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:26.844938 13530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b: No bootstrap required, opened a new log
I20260812 19:01:26.848261 13530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } }
I20260812 19:01:26.848471 13530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:26.848536 13530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc582ee72be41ad863cb9ea4d95691b, State: Initialized, Role: FOLLOWER
I20260812 19:01:26.848955 13530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } }
I20260812 19:01:26.849390 13752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } } }
I20260812 19:01:26.849568 13752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:01:26.850186 13530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [sys.catalog]: configured and running, proceeding with master startup.
W20260812 19:01:26.854878 13765 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:01:26.855002 13765 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:01:26.860422 13457 master_runner.cc:263] No leader master found from master 8cc582ee72be41ad863cb9ea4d95691b
I20260812 19:01:26.860489 13457 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:01:26.876133 13752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:01:26.876291 13752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } }
I20260812 19:01:26.877210 13752 leader_election.cc:288] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb7a38b3de8e4cc085622be4cbf29301 (127.29.112.253:46609), c046156e81444f6f92e0c1bd539c3b71 (127.29.112.252:35605)
I20260812 19:01:26.877755 13674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cc582ee72be41ad863cb9ea4d95691b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c046156e81444f6f92e0c1bd539c3b71" is_pre_election: true
I20260812 19:01:26.878105 13674 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cc582ee72be41ad863cb9ea4d95691b in term 0.
I20260812 19:01:26.878526 13475 leader_election.cc:302] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cc582ee72be41ad863cb9ea4d95691b, c046156e81444f6f92e0c1bd539c3b71; no voters:
I20260812 19:01:26.878803 13752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:01:26.878880 13752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:01:26.878930 13752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:26.880241 13752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } }
I20260812 19:01:26.880553 13752 leader_election.cc:288] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [CANDIDATE]: Term 1 election: Requested vote from peers eb7a38b3de8e4cc085622be4cbf29301 (127.29.112.253:46609), c046156e81444f6f92e0c1bd539c3b71 (127.29.112.252:35605)
I20260812 19:01:26.882745 13585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cc582ee72be41ad863cb9ea4d95691b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7a38b3de8e4cc085622be4cbf29301"
I20260812 19:01:26.882944 13586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cc582ee72be41ad863cb9ea4d95691b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7a38b3de8e4cc085622be4cbf29301" is_pre_election: true
I20260812 19:01:26.882946 13585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:26.884523 13585 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cc582ee72be41ad863cb9ea4d95691b in term 1.
I20260812 19:01:26.885061 13674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8cc582ee72be41ad863cb9ea4d95691b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c046156e81444f6f92e0c1bd539c3b71"
I20260812 19:01:26.885169 13674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:26.886301 13674 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cc582ee72be41ad863cb9ea4d95691b in term 1.
I20260812 19:01:26.886577 13475 leader_election.cc:302] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cc582ee72be41ad863cb9ea4d95691b, c046156e81444f6f92e0c1bd539c3b71; no voters:
I20260812 19:01:26.887351 13752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:01:26.887581 13752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [term 1 LEADER]: Becoming Leader. State: Replica: 8cc582ee72be41ad863cb9ea4d95691b, State: Running, Role: LEADER
I20260812 19:01:26.887903 13752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } }
I20260812 19:01:26.889567 13752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cc582ee72be41ad863cb9ea4d95691b. Latest consensus state: current_term: 1 leader_uuid: "8cc582ee72be41ad863cb9ea4d95691b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } } }
I20260812 19:01:26.889668 13752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:26.890053 13769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:01:26.890933 13769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:01:26.896997 13674 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [term 1 FOLLOWER]: Refusing update from remote peer 8cc582ee72be41ad863cb9ea4d95691b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:01:26.897569 13752 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:01:26.900894 13718 sys_catalog.cc:455] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cc582ee72be41ad863cb9ea4d95691b. Latest consensus state: current_term: 1 leader_uuid: "8cc582ee72be41ad863cb9ea4d95691b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } } }
I20260812 19:01:26.901010 13718 sys_catalog.cc:458] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:01:26.904120 13585 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [term 1 FOLLOWER]: Refusing update from remote peer 8cc582ee72be41ad863cb9ea4d95691b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:01:26.904578 13752 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:01:26.907601 13734 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cc582ee72be41ad863cb9ea4d95691b. Latest consensus state: current_term: 1 leader_uuid: "8cc582ee72be41ad863cb9ea4d95691b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } } }
I20260812 19:01:26.907708 13734 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:01:26.916117 13767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8cc582ee72be41ad863cb9ea4d95691b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } } }
I20260812 19:01:26.916242 13767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:26.917039 13734 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8cc582ee72be41ad863cb9ea4d95691b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } } }
I20260812 19:01:26.917146 13734 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:01:26.917599 13771 mvcc.cc:205] Tried to move back new op lower bound from 7317755031123759104 to 7317755031095963648. Current Snapshot: MvccSnapshot[applied={T|T < 7317755031123759104}]
I20260812 19:01:26.918439 13752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8cc582ee72be41ad863cb9ea4d95691b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } } }
I20260812 19:01:26.918548 13752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:26.919551 13718 sys_catalog.cc:455] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8cc582ee72be41ad863cb9ea4d95691b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cc582ee72be41ad863cb9ea4d95691b" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44841 } } peers { permanent_uuid: "eb7a38b3de8e4cc085622be4cbf29301" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 46609 } } peers { permanent_uuid: "c046156e81444f6f92e0c1bd539c3b71" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 35605 } } }
I20260812 19:01:26.919688 13718 sys_catalog.cc:458] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:01:26.920701 13775 mvcc.cc:205] Tried to move back new op lower bound from 7317755031123759104 to 7317755031095963648. Current Snapshot: MvccSnapshot[applied={T|T < 7317755031123759104}]
I20260812 19:01:26.920862 13772 mvcc.cc:205] Tried to move back new op lower bound from 7317755031123759104 to 7317755031095963648. Current Snapshot: MvccSnapshot[applied={T|T < 7317755031123759104}]
I20260812 19:01:26.923763 13769 catalog_manager.cc:1347] Generated new cluster ID: 51ba879398f3473ca4db5827f0ac2a5a
I20260812 19:01:26.923820 13769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:01:26.952704 13769 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:01:26.953873 13769 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:01:26.987494 13769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cc582ee72be41ad863cb9ea4d95691b: Generated new TSK 0
I20260812 19:01:26.988348 13769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:01:27.820829 13731 catalog_manager.cc:1259] Loaded cluster ID: 51ba879398f3473ca4db5827f0ac2a5a
I20260812 19:01:27.820909 13731 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71: loading cluster ID for follower catalog manager: success
I20260812 19:01:27.822715 13731 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71: acquiring CA information for follower catalog manager: success
I20260812 19:01:27.823834 13731 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c046156e81444f6f92e0c1bd539c3b71: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:01:27.839782 13747 catalog_manager.cc:1259] Loaded cluster ID: 51ba879398f3473ca4db5827f0ac2a5a
I20260812 19:01:27.839857 13747 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301: loading cluster ID for follower catalog manager: success
I20260812 19:01:27.841378 13747 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301: acquiring CA information for follower catalog manager: success
I20260812 19:01:27.842509 13747 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eb7a38b3de8e4cc085622be4cbf29301: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:01:28.054826 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 13457
I20260812 19:01:28.063542 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 13537
I20260812 19:01:28.070990 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 13623
2026-08-12T19:01:28Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3095 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:01:28 dist-test-slave-6nmv krb5kdc[13788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:01:28 dist-test-slave-6nmv krb5kdc[13788](info): set up 2 sockets
Aug 12 19:01:28 dist-test-slave-6nmv krb5kdc[13788](info): commencing operation
krb5kdc: starting...
W20260812 19:01:29.146431 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:01:29 dist-test-slave-6nmv krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:01:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:01:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:01:29.268882 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:45191
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:35137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:45191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:29.376403 13801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:29.376693 13801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:29.376762 13801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:29.380688 13801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:01:29.380766 13801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:29.380801 13801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:29.380849 13801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:01:29.380887 13801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:01:29.385571 13801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:35137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:45191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:45191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:29.386595 13801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:29.387614 13801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:29.395691 13808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:29.395678 13806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:29.395681 13801 server_base.cc:1034] running on GCE node
W20260812 19:01:29.395887 13810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:29.396435 13801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:29.397538 13801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:29.398718 13801 hybrid_clock.cc:648] HybridClock initialized: now 1786561289398688 us; error 44 us; skew 500 ppm
Aug 12 19:01:29 dist-test-slave-6nmv krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561289, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:29.401378 13801 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:01:29.402420 13801 webserver.cc:460] Webserver started at http://127.29.112.254:37593/ using document root <none> and password file <none>
I20260812 19:01:29.402948 13801 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:29.403040 13801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:29.403239 13801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:29.405212 13801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "e1e57f5776564b7f88f87d80734d3172"
format_stamp: "Formatted at 2026-08-12 19:01:29 on dist-test-slave-6nmv"
server_key: "16453224393cb67ac24ed3058443a329"
server_key_iv: "7b857a92d4d8e655c8154fb2c3fb09d0"
server_key_version: "encryptionkey@0"
I20260812 19:01:29.405751 13801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "e1e57f5776564b7f88f87d80734d3172"
format_stamp: "Formatted at 2026-08-12 19:01:29 on dist-test-slave-6nmv"
server_key: "16453224393cb67ac24ed3058443a329"
server_key_iv: "7b857a92d4d8e655c8154fb2c3fb09d0"
server_key_version: "encryptionkey@0"
I20260812 19:01:29.409188 13801 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:01:29.411461 13821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:29.412554 13801 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 19:01:29.412686 13801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "e1e57f5776564b7f88f87d80734d3172"
format_stamp: "Formatted at 2026-08-12 19:01:29 on dist-test-slave-6nmv"
server_key: "16453224393cb67ac24ed3058443a329"
server_key_iv: "7b857a92d4d8e655c8154fb2c3fb09d0"
server_key_version: "encryptionkey@0"
I20260812 19:01:29.412789 13801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:29.436851 13801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:29.439684 13801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:29.439903 13801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:29.446499 13801 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:45191
I20260812 19:01:29.446535 13879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:45191 every 8 connection(s)
I20260812 19:01:29.447397 13801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:01:29.450076 13880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:29.455240 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 13801
I20260812 19:01:29.455369 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:01:29.455415 13880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172: Bootstrap starting.
I20260812 19:01:29.455711 30147 external_mini_cluster.cc:1428] Setting key 3c6f180e13169c50e864f92fae698903
I20260812 19:01:29.457975 13880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:29.458703 13880 log.cc:824] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:29.460446 13880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172: No bootstrap required, opened a new log
Aug 12 19:01:29 dist-test-slave-6nmv krb5kdc[13788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:29.464457 13880 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e57f5776564b7f88f87d80734d3172" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45191 } }
I20260812 19:01:29.464701 13880 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:29.464766 13880 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1e57f5776564b7f88f87d80734d3172, State: Initialized, Role: FOLLOWER
I20260812 19:01:29.465226 13880 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e57f5776564b7f88f87d80734d3172" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45191 } }
I20260812 19:01:29.465394 13880 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:01:29.465462 13880 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:01:29.465584 13880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:29.466734 13880 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e57f5776564b7f88f87d80734d3172" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45191 } }
I20260812 19:01:29.467063 13880 leader_election.cc:302] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1e57f5776564b7f88f87d80734d3172; no voters:
I20260812 19:01:29.467417 13880 leader_election.cc:288] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:01:29.467514 13886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:01:29.467788 13886 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [term 1 LEADER]: Becoming Leader. State: Replica: e1e57f5776564b7f88f87d80734d3172, State: Running, Role: LEADER
I20260812 19:01:29.468171 13886 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e57f5776564b7f88f87d80734d3172" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45191 } }
I20260812 19:01:29.468917 13880 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:01:29.470031 13887 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1e57f5776564b7f88f87d80734d3172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e57f5776564b7f88f87d80734d3172" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45191 } } }
I20260812 19:01:29.470139 13887 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:29.470360 13887 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1e57f5776564b7f88f87d80734d3172. Latest consensus state: current_term: 1 leader_uuid: "e1e57f5776564b7f88f87d80734d3172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e57f5776564b7f88f87d80734d3172" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 45191 } } }
I20260812 19:01:29.470424 13887 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172 [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:29.470577 13883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:29.457008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41474 (local address 127.29.112.254:45191)
0812 19:01:29.457408 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:29.457422 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:29.457454 (+ 32us) server_negotiation.cc:407] Connection header received
0812 19:01:29.458146 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:29.458165 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:29.458472 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:29.458823 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:29.459619 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:29.460494 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:29.461538 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:29.461982 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:29.462090 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:29.464650 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:29.464656 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:29.464671 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:29.464701 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:29.466743 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:29.467200 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:29.467206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:29.467211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:29.467409 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:29.467781 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:29.467784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:29.467786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:29.468347 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:29.468560 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:29.468993 (+ 433us) server_negotiation.cc:299] Negotiation successful
0812 19:01:29.469469 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":126,"threads_started":1}
I20260812 19:01:29.473479 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 19:01:29.473850 13902 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:01:29.473922 13902 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:01:29.473994 13896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:01:29.474766 13896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:01:29.479930 13896 catalog_manager.cc:1347] Generated new cluster ID: 99c06c49a28c4011943446469d5aa352
I20260812 19:01:29.480010 13896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:01:29.494052 13896 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:01:29.495298 13896 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:01:29.499954 13896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1e57f5776564b7f88f87d80734d3172: Generated new TSK 0
I20260812 19:01:29.500737 13896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:01:29.529659 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 13801
2026-08-12T19:01:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1454 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:01:29 dist-test-slave-6nmv krb5kdc[13909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:01:29 dist-test-slave-6nmv krb5kdc[13909](info): set up 2 sockets
Aug 12 19:01:29 dist-test-slave-6nmv krb5kdc[13909](info): commencing operation
krb5kdc: starting...
W20260812 19:01:30.619290 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:01:30 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:01:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:01:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:01:30.738976 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38735
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:38735
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:30.842933 13927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:30.843175 13927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:30.843210 13927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:30.846899 13927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:01:30.846946 13927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:30.846962 13927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:30.846976 13927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:01:30.846989 13927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:01:30.851842 13927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:38735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:30.852707 13927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:30.853648 13927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:30.861492 13935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:30.861505 13933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:30.861513 13932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:30.861558 13927 server_base.cc:1034] running on GCE node
I20260812 19:01:30.862365 13927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:30.863436 13927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:30.864622 13927 hybrid_clock.cc:648] HybridClock initialized: now 1786561290864611 us; error 25 us; skew 500 ppm
Aug 12 19:01:30 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561290, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:30.874689 13927 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:01:30.875578 13927 webserver.cc:460] Webserver started at http://127.29.112.254:45293/ using document root <none> and password file <none>
I20260812 19:01:30.876118 13927 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:30.876207 13927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:30.876427 13927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:30.878513 13927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "e57e976ec05445748ab9d223268b7f7c"
format_stamp: "Formatted at 2026-08-12 19:01:30 on dist-test-slave-6nmv"
server_key: "9437ddf1f0fc1a41cc4e693ceaa51462"
server_key_iv: "173563929a9450537230a81152c9a56d"
server_key_version: "encryptionkey@0"
I20260812 19:01:30.879083 13927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "e57e976ec05445748ab9d223268b7f7c"
format_stamp: "Formatted at 2026-08-12 19:01:30 on dist-test-slave-6nmv"
server_key: "9437ddf1f0fc1a41cc4e693ceaa51462"
server_key_iv: "173563929a9450537230a81152c9a56d"
server_key_version: "encryptionkey@0"
I20260812 19:01:30.882687 13927 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:01:30.884837 13942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:30.886056 13927 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:01:30.886181 13927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "e57e976ec05445748ab9d223268b7f7c"
format_stamp: "Formatted at 2026-08-12 19:01:30 on dist-test-slave-6nmv"
server_key: "9437ddf1f0fc1a41cc4e693ceaa51462"
server_key_iv: "173563929a9450537230a81152c9a56d"
server_key_version: "encryptionkey@0"
I20260812 19:01:30.886286 13927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:30.907083 13927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:30.910121 13927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:30.910339 13927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:30.917752 13927 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:38735
I20260812 19:01:30.918066 14009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:38735 every 8 connection(s)
I20260812 19:01:30.918900 13927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:01:30.921600 14010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:30.925148 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 13927
I20260812 19:01:30.925282 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:01:30.925560 30147 external_mini_cluster.cc:1428] Setting key be1df7dbdad6306be6644316c08f3e48
I20260812 19:01:30.927639 14010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Bootstrap starting.
I20260812 19:01:30.930217 14010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:30.930989 14010 log.cc:824] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:30.933197 14010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: No bootstrap required, opened a new log
Aug 12 19:01:30 dist-test-slave-6nmv krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:30.938040 14010 raft_consensus.cc:348] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:30.938264 14010 raft_consensus.cc:374] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:30.938302 14010 raft_consensus.cc:729] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e57e976ec05445748ab9d223268b7f7c, State: Initialized, Role: FOLLOWER
I20260812 19:01:30.938796 14010 consensus_queue.cc:260] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:30.938971 14010 raft_consensus.cc:388] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:01:30.939075 14010 raft_consensus.cc:482] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:01:30.939206 14010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:30.940224 14010 raft_consensus.cc:504] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:30.940644 14010 leader_election.cc:302] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e57e976ec05445748ab9d223268b7f7c; no voters:
I20260812 19:01:30.940989 14010 leader_election.cc:288] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:01:30.941092 14015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:01:30.941335 14015 raft_consensus.cc:686] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 1 LEADER]: Becoming Leader. State: Replica: e57e976ec05445748ab9d223268b7f7c, State: Running, Role: LEADER
I20260812 19:01:30.941718 14015 consensus_queue.cc:237] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:30.941854 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:30.927248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59908 (local address 127.29.112.254:38735)
0812 19:01:30.927677 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:30.927693 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:30.927732 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:01:30.928472 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:30.928489 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:30.928826 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:30.929118 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:30.930058 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:30.931234 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:30.932462 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:30.932960 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:30.933091 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:30.936095 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:30.936101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:30.936113 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:30.936139 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:30.938028 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:30.938713 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:30.938719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:30.938724 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:30.938850 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:30.939220 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:30.939229 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:30.939231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:30.939747 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:30.939987 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:30.940314 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 19:01:30.940659 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":274,"thread_start_us":149,"threads_started":1}
I20260812 19:01:30.942277 14010 sys_catalog.cc:564] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:01:30.943080 14015 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e57e976ec05445748ab9d223268b7f7c. Latest consensus state: current_term: 1 leader_uuid: "e57e976ec05445748ab9d223268b7f7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } } }
I20260812 19:01:30.943270 14015 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:30.943537 14016 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e57e976ec05445748ab9d223268b7f7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } } }
I20260812 19:01:30.943645 14016 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:30.943745 14027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:01:30.947335 14027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:01:30.953377 14027 catalog_manager.cc:1347] Generated new cluster ID: c5bd8e85d8984411a18c796a95672c4f
I20260812 19:01:30.953460 14027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:01:30.959424 14027 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:01:30.960350 14027 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:01:30.967749 14027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Generated new TSK 0
I20260812 19:01:30.968484 14027 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 19:01:31.005100 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38735
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:31.124914 14040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:31.125142 14040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:31.125183 14040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:31.128814 14040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:31.128872 14040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:31.128926 14040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:01:31.134382 14040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:31.135267 14040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:31.136142 14040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:31.145509 14046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:31.145551 14048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:31.145552 14045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:31.145874 14040 server_base.cc:1034] running on GCE node
I20260812 19:01:31.146202 14040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:31.146776 14040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:31.147943 14040 hybrid_clock.cc:648] HybridClock initialized: now 1786561291147922 us; error 37 us; skew 500 ppm
Aug 12 19:01:31 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561291, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:31.150676 14040 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:01:31.151484 14040 webserver.cc:460] Webserver started at http://127.29.112.193:41983/ using document root <none> and password file <none>
I20260812 19:01:31.152043 14040 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:31.152103 14040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:31.152379 14040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:31.154479 14040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "f6dd4bcff003483ba673140f1aedcf08"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "2e55d1ad4d266d0e69e344aa45600e4f"
server_key_iv: "4a6afa5abb686c2a06e297b55ddea661"
server_key_version: "encryptionkey@0"
I20260812 19:01:31.155028 14040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "f6dd4bcff003483ba673140f1aedcf08"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "2e55d1ad4d266d0e69e344aa45600e4f"
server_key_iv: "4a6afa5abb686c2a06e297b55ddea661"
server_key_version: "encryptionkey@0"
I20260812 19:01:31.158542 14040 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:01:31.160941 14055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:31.162163 14040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:01:31.162274 14040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "f6dd4bcff003483ba673140f1aedcf08"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "2e55d1ad4d266d0e69e344aa45600e4f"
server_key_iv: "4a6afa5abb686c2a06e297b55ddea661"
server_key_version: "encryptionkey@0"
I20260812 19:01:31.162396 14040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:31.188448 14040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:31.191736 14040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:31.191967 14040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:31.192586 14040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:31.193434 14040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:31.193480 14040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:31.193548 14040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:31.193581 14040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:31.202505 14040 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:40085
I20260812 19:01:31.202529 14192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:40085 every 8 connection(s)
I20260812 19:01:31.203394 14040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
Aug 12 19:01:31 dist-test-slave-6nmv krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561291, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:31.213205 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 14040
I20260812 19:01:31.213346 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 19:01:31.213650 30147 external_mini_cluster.cc:1428] Setting key 047ffb87670c472443c96e806f4a2465
I20260812 19:01:31.217555 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.205256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:59223 (local address 127.29.112.254:38735)
0812 19:01:31.205402 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:31.205408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:31.206337 (+ 929us) server_negotiation.cc:407] Connection header received
0812 19:01:31.207631 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:31.207634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:31.207678 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:31.207751 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:31.208869 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.209419 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.210729 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.211052 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.211135 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.213613 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:31.213616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:31.213618 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:31.213641 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:31.215452 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:31.215936 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:31.215942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:31.215946 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:31.216047 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:31.216358 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:31.216364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:31.216369 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:31.216624 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:31.216755 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:31.217189 (+ 434us) server_negotiation.cc:299] Negotiation successful
0812 19:01:31.217335 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:01:31.218613 14195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.205654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:59223)
0812 19:01:31.206191 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:31.206235 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:31.207431 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:31.207900 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:31.207906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:31.208246 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:31.208722 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.208736 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.209547 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.209553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.210607 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.210617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.211179 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.211182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.211308 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.211866 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:31.211882 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:31.213441 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:31.215575 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:31.215579 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:31.215589 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:31.215836 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:31.216138 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:31.216140 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:31.216141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:31.216268 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:31.216743 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:31.216747 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:31.216962 (+ 215us) client_negotiation.cc:769] Sending connection context
0812 19:01:31.217111 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 19:01:31.217473 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":161,"threads_started":1}
I20260812 19:01:31.219926 14193 heartbeater.cc:347] Connected to a master server at 127.29.112.254:38735
I20260812 19:01:31.220180 14193 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:31.220747 14193 heartbeater.cc:511] Master 127.29.112.254:38735 requested a full tablet report, sending...
I20260812 19:01:31.222373 13964 ts_manager.cc:194] Registered new tserver with Master: f6dd4bcff003483ba673140f1aedcf08 (127.29.112.193:40085)
I20260812 19:01:31.223493 13964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:59223
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 19:01:31.242925 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38735
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:31.350705 14198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:31.350984 14198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:31.351061 14198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:31.354996 14198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:31.355073 14198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:31.355158 14198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:01:31.360127 14198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:31.361063 14198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:31.361999 14198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:31.371220 14204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:31.371263 14203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:31.371383 14198 server_base.cc:1034] running on GCE node
W20260812 19:01:31.371230 14206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:31.371892 14198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:31.372526 14198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:31.373714 14198 hybrid_clock.cc:648] HybridClock initialized: now 1786561291373686 us; error 46 us; skew 500 ppm
Aug 12 19:01:31 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561291, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:31.376619 14198 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:01:31.377506 14198 webserver.cc:460] Webserver started at http://127.29.112.194:46337/ using document root <none> and password file <none>
I20260812 19:01:31.378120 14198 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:31.378212 14198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:31.378446 14198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:31.381065 14198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "a91f7679cd6a4d71949e6eace4c5c66e"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "70855a7eae6ca3513133ab21246542ed"
server_key_iv: "13ac2983a1784b0c8e8b2390eb9932cc"
server_key_version: "encryptionkey@0"
I20260812 19:01:31.381652 14198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "a91f7679cd6a4d71949e6eace4c5c66e"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "70855a7eae6ca3513133ab21246542ed"
server_key_iv: "13ac2983a1784b0c8e8b2390eb9932cc"
server_key_version: "encryptionkey@0"
I20260812 19:01:31.385236 14198 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 19:01:31.387693 14214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:31.388916 14198 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 19:01:31.389055 14198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "a91f7679cd6a4d71949e6eace4c5c66e"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "70855a7eae6ca3513133ab21246542ed"
server_key_iv: "13ac2983a1784b0c8e8b2390eb9932cc"
server_key_version: "encryptionkey@0"
I20260812 19:01:31.389161 14198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:31.423900 14198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:31.427265 14198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:31.427472 14198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:31.428058 14198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:31.428920 14198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:31.428992 14198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:31.429066 14198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:31.429121 14198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:31.437871 14198 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:37635
I20260812 19:01:31.437911 14344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:37635 every 8 connection(s)
I20260812 19:01:31.438751 14198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:01:31.439731 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 14198
I20260812 19:01:31.439842 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 19:01:31.440140 30147 external_mini_cluster.cc:1428] Setting key 5aaf70548446897b1b19810b0e4f68c7
Aug 12 19:01:31 dist-test-slave-6nmv krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561291, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
I20260812 19:01:31.454051 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.440653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:47055 (local address 127.29.112.254:38735)
0812 19:01:31.440816 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:31.440822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:31.441727 (+ 905us) server_negotiation.cc:407] Connection header received
0812 19:01:31.443693 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:31.443697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:31.443745 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:31.443837 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:31.445152 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.446038 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.447132 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.447559 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.447695 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.450336 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:31.450339 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:31.450342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:31.450372 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:31.451816 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:31.452435 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:31.452437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:31.452439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:31.452527 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:31.452825 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:31.452827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:31.452828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:31.453040 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:31.453121 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:31.453675 (+ 554us) server_negotiation.cc:299] Negotiation successful
0812 19:01:31.453814 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:01:31.454900 14348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.441073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:47055)
0812 19:01:31.441557 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:31.441613 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:31.443476 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:31.444113 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:31.444120 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:31.444553 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:31.445012 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.445025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.446157 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.446159 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.447012 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.447021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.447717 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.447719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.447941 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.448629 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:31.448663 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:31.450154 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:31.451921 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:31.451926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:31.451936 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:31.452336 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:31.452609 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:31.452611 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:31.452612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:31.452746 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:31.453172 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:31.453182 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:31.453475 (+ 293us) client_negotiation.cc:769] Sending connection context
0812 19:01:31.453626 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 19:01:31.454029 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":113,"threads_started":1}
I20260812 19:01:31.456079 14345 heartbeater.cc:347] Connected to a master server at 127.29.112.254:38735
I20260812 19:01:31.456357 14345 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:31.456930 14345 heartbeater.cc:511] Master 127.29.112.254:38735 requested a full tablet report, sending...
I20260812 19:01:31.458017 13964 ts_manager.cc:194] Registered new tserver with Master: a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635)
I20260812 19:01:31.458642 13964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:47055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 19:01:31.468889 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38735
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:31.575667 14351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:31.575996 14351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:31.576074 14351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:31.580183 14351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:31.580272 14351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:31.580363 14351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:01:31.585268 14351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:31.586371 14351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:31.587451 14351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:31.596730 14357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:31.596822 14351 server_base.cc:1034] running on GCE node
W20260812 19:01:31.596743 14358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:31.596743 14361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:31.597321 14351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:31.597970 14351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:31.599125 14351 hybrid_clock.cc:648] HybridClock initialized: now 1786561291599112 us; error 24 us; skew 500 ppm
Aug 12 19:01:31 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561291, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:31.602010 14351 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:01:31.602873 14351 webserver.cc:460] Webserver started at http://127.29.112.195:39275/ using document root <none> and password file <none>
I20260812 19:01:31.603410 14351 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:31.603466 14351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:31.603685 14351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:31.605593 14351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "e202ab34ffce4c42a67936b923e37188"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "ac0d19662f4bb47daa93a54efa19d9ad"
server_key_iv: "6e8f01cb85814f4e4323397b97e60269"
server_key_version: "encryptionkey@0"
I20260812 19:01:31.606201 14351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "e202ab34ffce4c42a67936b923e37188"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "ac0d19662f4bb47daa93a54efa19d9ad"
server_key_iv: "6e8f01cb85814f4e4323397b97e60269"
server_key_version: "encryptionkey@0"
I20260812 19:01:31.609745 14351 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:01:31.612021 14369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:31.613147 14351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:01:31.613270 14351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "e202ab34ffce4c42a67936b923e37188"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "ac0d19662f4bb47daa93a54efa19d9ad"
server_key_iv: "6e8f01cb85814f4e4323397b97e60269"
server_key_version: "encryptionkey@0"
I20260812 19:01:31.613396 14351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:31.623780 14351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:31.626740 14351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:31.626940 14351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:31.627461 14351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:31.628221 14351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:31.628289 14351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:31.628352 14351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:31.628382 14351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:31.637534 14351 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:39655
I20260812 19:01:31.637564 14501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:39655 every 8 connection(s)
I20260812 19:01:31.638442 14351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:01:31.645159 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 14351
I20260812 19:01:31.645272 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 19:01:31.645556 30147 external_mini_cluster.cc:1428] Setting key 8627334c05619e5780b98f64d033f387
Aug 12 19:01:31 dist-test-slave-6nmv krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561291, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:31.652381 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.640231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:40177 (local address 127.29.112.254:38735)
0812 19:01:31.640365 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:31.640369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:31.641124 (+ 755us) server_negotiation.cc:407] Connection header received
0812 19:01:31.642491 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:31.642494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:31.642558 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:31.642700 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:31.643876 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.644394 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.645864 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.646083 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.646212 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.649041 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:31.649045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:31.649048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:31.649080 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:31.650361 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:31.650865 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:31.650869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:31.650871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:31.650958 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:31.651300 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:31.651304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:31.651306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:31.651490 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:31.651597 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:31.652087 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 19:01:31.652212 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:01:31.653120 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.640537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:40177)
0812 19:01:31.640974 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:31.641013 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:31.642299 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:31.642855 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:31.642865 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:31.643268 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:31.643705 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.643718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.644520 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.644523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.645684 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.645696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.646251 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.646254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.646470 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.647173 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:31.647193 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:31.648855 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:31.650492 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:31.650497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:31.650511 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:31.650761 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:31.651058 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:31.651061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:31.651063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:31.651206 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:31.651594 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:31.651600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:31.651860 (+ 260us) client_negotiation.cc:769] Sending connection context
0812 19:01:31.652043 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 19:01:31.652346 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260812 19:01:31.654183 14502 heartbeater.cc:347] Connected to a master server at 127.29.112.254:38735
I20260812 19:01:31.654433 14502 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:31.654968 14502 heartbeater.cc:511] Master 127.29.112.254:38735 requested a full tablet report, sending...
I20260812 19:01:31.655961 13964 ts_manager.cc:194] Registered new tserver with Master: e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655)
I20260812 19:01:31.656503 13964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:40177
I20260812 19:01:31.659590 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:01:31.670312 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.661752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59922 (local address 127.29.112.254:38735)
0812 19:01:31.661929 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:31.661935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:31.662092 (+ 157us) server_negotiation.cc:407] Connection header received
0812 19:01:31.662180 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:31.662182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:31.662217 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:31.662277 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:31.663182 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.663910 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.665160 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.665397 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.665524 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.667208 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:31.667212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:31.667215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:31.667241 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:31.668493 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:31.668922 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:31.668926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:31.668928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:31.669029 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:31.669314 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:31.669318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:31.669319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:31.669562 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:31.669701 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:31.670026 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 19:01:31.670154 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 19:01:31.673532 13964 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:01:31.676744 13964 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:01:31.692387 14519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.686515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44006 (local address 127.29.112.195:39655)
0812 19:01:31.686869 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:31.686882 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:31.687334 (+ 452us) server_negotiation.cc:407] Connection header received
0812 19:01:31.687610 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:31.687615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:31.687777 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:31.687898 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:31.688353 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.689151 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.690801 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.691604 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.691756 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.691893 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:31.692136 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 19:01:31.692228 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":121,"threads_started":1}
I20260812 19:01:31.692379 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.686236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39655 (local address 127.0.0.1:44006)
0812 19:01:31.687139 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:31.687151 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:31.687466 (+ 315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:31.688075 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:31.688077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:31.688103 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:31.688231 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.688239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.689288 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.689292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.690689 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.690699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.691755 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.691758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.691918 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.691979 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 19:01:31.692105 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 19:01:31.692228 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":131,"spinlock_wait_cycles":482432,"thread_start_us":123,"threads_started":1}
I20260812 19:01:31.693058 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.685878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:37635 (local address 127.0.0.1:34518)
0812 19:01:31.687049 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:31.687065 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:31.687401 (+ 336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:31.688019 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:31.688022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:31.688042 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:31.688198 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.688205 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.689664 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.689667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.691082 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.691091 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.692239 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.692244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.692345 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.692376 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 19:01:31.692440 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 19:01:31.692880 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1091,"mutex_wait_us":73,"spinlock_wait_cycles":2816,"thread_start_us":108,"threads_started":1}
I20260812 19:01:31.693341 14515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.685706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:40085 (local address 127.0.0.1:33950)
0812 19:01:31.686992 (+ 1286us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:31.687046 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:31.687184 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:31.687795 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:31.687801 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:31.687825 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:31.688010 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.688020 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.690099 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.690102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.691369 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.691380 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.692757 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.692761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.692901 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.692927 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 19:01:31.693060 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 19:01:31.693175 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1100,"mutex_wait_us":119,"thread_start_us":120,"threads_started":1}
I20260812 19:01:31.694061 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.685950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33950 (local address 127.29.112.193:40085)
0812 19:01:31.686677 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:31.686698 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:31.687319 (+ 621us) server_negotiation.cc:407] Connection header received
0812 19:01:31.687406 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:31.687412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:31.687628 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:31.687778 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:31.688171 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.689462 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.691498 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.692451 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.692572 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.692692 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:31.693757 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0812 19:01:31.693907 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":148,"threads_started":1}
I20260812 19:01:31.694113 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.686622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.29.112.194:37635)
0812 19:01:31.687004 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:31.687016 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:31.687399 (+ 383us) server_negotiation.cc:407] Connection header received
0812 19:01:31.687590 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:31.687596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:31.687804 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:31.688022 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:31.688317 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.689462 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.691220 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.692038 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.693649 (+ 1611us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.693795 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:31.693877 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 19:01:31.693965 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":101,"threads_started":1}
I20260812 19:01:31.695799 14426 tablet_service.cc:1467] Processing CreateTablet for tablet 9406c4d78dbf4ecd8ad1eab00f30b87c (DEFAULT_TABLE table=test-table [id=ef377b3db56d42a28832fa245f0fa526]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:31.696504 14269 tablet_service.cc:1467] Processing CreateTablet for tablet 9406c4d78dbf4ecd8ad1eab00f30b87c (DEFAULT_TABLE table=test-table [id=ef377b3db56d42a28832fa245f0fa526]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:31.696966 14426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9406c4d78dbf4ecd8ad1eab00f30b87c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:31.696892 14112 tablet_service.cc:1467] Processing CreateTablet for tablet 9406c4d78dbf4ecd8ad1eab00f30b87c (DEFAULT_TABLE table=test-table [id=ef377b3db56d42a28832fa245f0fa526]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:31.697491 14269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9406c4d78dbf4ecd8ad1eab00f30b87c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:31.697921 14112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9406c4d78dbf4ecd8ad1eab00f30b87c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:31.703606 14523 tablet_bootstrap.cc:492] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: Bootstrap starting.
I20260812 19:01:31.705099 14524 tablet_bootstrap.cc:492] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: Bootstrap starting.
I20260812 19:01:31.706161 14525 tablet_bootstrap.cc:492] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Bootstrap starting.
I20260812 19:01:31.706607 14523 tablet_bootstrap.cc:654] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:31.707563 14524 tablet_bootstrap.cc:654] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:31.707557 14523 log.cc:824] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:31.708326 14525 tablet_bootstrap.cc:654] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:31.708479 14524 log.cc:824] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:31.709582 14525 log.cc:824] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:31.710167 14523 tablet_bootstrap.cc:492] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: No bootstrap required, opened a new log
I20260812 19:01:31.710414 14523 ts_tablet_manager.cc:1397] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 19:01:31.710656 14524 tablet_bootstrap.cc:492] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: No bootstrap required, opened a new log
I20260812 19:01:31.710835 14524 ts_tablet_manager.cc:1397] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:01:31.711577 14525 tablet_bootstrap.cc:492] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: No bootstrap required, opened a new log
I20260812 19:01:31.711781 14525 ts_tablet_manager.cc:1397] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:01:31.713670 14523 raft_consensus.cc:348] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } }
I20260812 19:01:31.714015 14523 raft_consensus.cc:374] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:31.714097 14523 raft_consensus.cc:729] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a91f7679cd6a4d71949e6eace4c5c66e, State: Initialized, Role: FOLLOWER
I20260812 19:01:31.714568 14523 consensus_queue.cc:260] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } }
I20260812 19:01:31.714656 14524 raft_consensus.cc:348] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } }
I20260812 19:01:31.714926 14524 raft_consensus.cc:374] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:31.715036 14524 raft_consensus.cc:729] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e202ab34ffce4c42a67936b923e37188, State: Initialized, Role: FOLLOWER
I20260812 19:01:31.715494 14524 consensus_queue.cc:260] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } }
I20260812 19:01:31.715544 14525 raft_consensus.cc:348] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } }
I20260812 19:01:31.715692 14523 ts_tablet_manager.cc:1428] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:01:31.715816 14525 raft_consensus.cc:374] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:31.715932 14525 raft_consensus.cc:729] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6dd4bcff003483ba673140f1aedcf08, State: Initialized, Role: FOLLOWER
I20260812 19:01:31.716043 14345 heartbeater.cc:503] Master 127.29.112.254:38735 was elected leader, sending a full tablet report...
I20260812 19:01:31.716385 14525 consensus_queue.cc:260] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } }
I20260812 19:01:31.716575 14524 ts_tablet_manager.cc:1428] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:01:31.717135 14193 heartbeater.cc:503] Master 127.29.112.254:38735 was elected leader, sending a full tablet report...
I20260812 19:01:31.717199 14502 heartbeater.cc:503] Master 127.29.112.254:38735 was elected leader, sending a full tablet report...
I20260812 19:01:31.718474 14525 ts_tablet_manager.cc:1428] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260812 19:01:31.889492 14503 tablet.cc:2369] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:01:31.939901 14347 tablet.cc:2369] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 19:01:31.954798 14194 tablet.cc:2369] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:01:31.991242 14530 raft_consensus.cc:482] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:01:31.991472 14530 raft_consensus.cc:504] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } }
I20260812 19:01:31.992825 14530 leader_election.cc:288] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6dd4bcff003483ba673140f1aedcf08 (127.29.112.193:40085), e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655)
I20260812 19:01:31.997592 14536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.993158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39655 (local address 127.29.112.194:56387)
0812 19:01:31.993582 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:31.993598 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:31.993707 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:31.994048 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:31.994052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:31.994075 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:31.994234 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.994241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.995071 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.995075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.996172 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.996182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.997151 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.997155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.997251 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.997277 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 19:01:31.997352 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 19:01:31.997436 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":70,"threads_started":1}
I20260812 19:01:31.997627 14519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.993504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:56387 (local address 127.29.112.195:39655)
0812 19:01:31.993653 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:31.993659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:31.993681 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:01:31.993876 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:31.993880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:31.993930 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:31.994015 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:31.994366 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.994953 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.996306 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.997015 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.997153 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.997217 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:31.997387 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 19:01:31.997470 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 19:01:31.997709 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.993789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:41745 (local address 127.29.112.193:40085)
0812 19:01:31.993936 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:31.993940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:31.993955 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:01:31.994010 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:31.994012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:31.994056 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:31.994124 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:31.994548 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.995402 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.996645 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.997047 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.997177 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.997228 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:31.997501 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 19:01:31.997575 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 19:01:31.998071 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:31.993051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:40085 (local address 127.29.112.194:41745)
0812 19:01:31.993302 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:31.993317 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:31.993450 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:31.994227 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:31.994229 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:31.994244 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:31.994373 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.994379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.995530 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.995534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.996526 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:31.996534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:31.997277 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:31.997280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:31.997374 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:31.997392 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 19:01:31.997914 (+ 522us) client_negotiation.cc:240] Negotiation successful
0812 19:01:31.997980 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":100,"threads_started":1}
I20260812 19:01:31.998621 14446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9406c4d78dbf4ecd8ad1eab00f30b87c" candidate_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e202ab34ffce4c42a67936b923e37188" is_pre_election: true
I20260812 19:01:31.998715 14137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9406c4d78dbf4ecd8ad1eab00f30b87c" candidate_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6dd4bcff003483ba673140f1aedcf08" is_pre_election: true
I20260812 19:01:31.998997 14446 raft_consensus.cc:2454] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a91f7679cd6a4d71949e6eace4c5c66e in term 0.
I20260812 19:01:31.999061 14137 raft_consensus.cc:2454] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a91f7679cd6a4d71949e6eace4c5c66e in term 0.
I20260812 19:01:31.999516 14215 leader_election.cc:302] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a91f7679cd6a4d71949e6eace4c5c66e, e202ab34ffce4c42a67936b923e37188; no voters:
I20260812 19:01:31.999854 14530 raft_consensus.cc:2781] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:01:31.999960 14530 raft_consensus.cc:482] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:01:32.000016 14530 raft_consensus.cc:3037] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:32.001013 14530 raft_consensus.cc:504] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } }
I20260812 19:01:32.001336 14530 leader_election.cc:288] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [CANDIDATE]: Term 1 election: Requested vote from peers f6dd4bcff003483ba673140f1aedcf08 (127.29.112.193:40085), e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655)
I20260812 19:01:32.001616 14137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9406c4d78dbf4ecd8ad1eab00f30b87c" candidate_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6dd4bcff003483ba673140f1aedcf08"
I20260812 19:01:32.001744 14446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9406c4d78dbf4ecd8ad1eab00f30b87c" candidate_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e202ab34ffce4c42a67936b923e37188"
I20260812 19:01:32.001765 14137 raft_consensus.cc:3037] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:32.001884 14446 raft_consensus.cc:3037] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:32.003052 14137 raft_consensus.cc:2454] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a91f7679cd6a4d71949e6eace4c5c66e in term 1.
I20260812 19:01:32.003052 14446 raft_consensus.cc:2454] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a91f7679cd6a4d71949e6eace4c5c66e in term 1.
I20260812 19:01:32.003334 14218 leader_election.cc:302] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a91f7679cd6a4d71949e6eace4c5c66e, f6dd4bcff003483ba673140f1aedcf08; no voters:
I20260812 19:01:32.003496 14530 raft_consensus.cc:2781] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:01:32.003738 14530 raft_consensus.cc:686] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 1 LEADER]: Becoming Leader. State: Replica: a91f7679cd6a4d71949e6eace4c5c66e, State: Running, Role: LEADER
I20260812 19:01:32.004103 14530 consensus_queue.cc:237] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } }
I20260812 19:01:32.007278 13964 catalog_manager.cc:5495] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e reported cstate change: term changed from 0 to 1, leader changed from <none> to a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194). New cstate: current_term: 1 leader_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } health_report { overall_health: UNKNOWN } } }
I20260812 19:01:32.055801 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.052259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34532 (local address 127.29.112.194:37635)
0812 19:01:32.052417 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:32.052422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:32.052439 (+ 17us) server_negotiation.cc:407] Connection header received
0812 19:01:32.052536 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:32.052538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:32.052594 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:32.052711 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:01:32.053000 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.053513 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.054390 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.054605 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.054711 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.055031 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:01:32.055432 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:01:32.055563 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:32.055594 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 19:01:32.055649 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 19:01:32.062331 14446 raft_consensus.cc:1261] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188 [term 1 FOLLOWER]: Refusing update from remote peer a91f7679cd6a4d71949e6eace4c5c66e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:01:32.062397 14137 raft_consensus.cc:1261] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Refusing update from remote peer a91f7679cd6a4d71949e6eace4c5c66e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:01:32.063082 14537 consensus_queue.cc:1035] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:01:32.063328 14530 consensus_queue.cc:1035] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:01:32.070241 14541 mvcc.cc:205] Tried to move back new op lower bound from 7317755052279693312 to 7317755052051451904. Current Snapshot: MvccSnapshot[applied={T|T < 7317755052279693312}]
I20260812 19:01:32.070362 14544 mvcc.cc:205] Tried to move back new op lower bound from 7317755052279693312 to 7317755052051451904. Current Snapshot: MvccSnapshot[applied={T|T < 7317755052279693312}]
I20260812 19:01:32.079582 13962 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:01:32.079769 13962 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:01:32.082526 13962 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Sending DeleteTablet for 3 replicas of tablet 9406c4d78dbf4ecd8ad1eab00f30b87c
I20260812 19:01:32.083299 14112 tablet_service.cc:1514] Processing DeleteTablet for tablet 9406c4d78dbf4ecd8ad1eab00f30b87c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:32 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:33950
I20260812 19:01:32.083410 14269 tablet_service.cc:1514] Processing DeleteTablet for tablet 9406c4d78dbf4ecd8ad1eab00f30b87c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:32 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:34518
I20260812 19:01:32.083420 14426 tablet_service.cc:1514] Processing DeleteTablet for tablet 9406c4d78dbf4ecd8ad1eab00f30b87c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:32 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:44006
I20260812 19:01:32.083880 14555 tablet_replica.cc:330] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: stopping tablet replica
I20260812 19:01:32.083899 14554 tablet_replica.cc:330] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: stopping tablet replica
I20260812 19:01:32.084175 14555 raft_consensus.cc:2229] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:01:32.084175 14554 raft_consensus.cc:2229] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:01:32.084200 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 14040
I20260812 19:01:32.084446 14555 raft_consensus.cc:2258] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:01:32.087232 14555 ts_tablet_manager.cc:1905] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:01:32.089613 14554 raft_consensus.cc:2258] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:01:32.090447 14555 ts_tablet_manager.cc:1918] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:01:32.090556 14555 log.cc:1196] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/9406c4d78dbf4ecd8ad1eab00f30b87c
I20260812 19:01:32.090883 14555 ts_tablet_manager.cc:1939] T 9406c4d78dbf4ecd8ad1eab00f30b87c P e202ab34ffce4c42a67936b923e37188: Deleting consensus metadata
I20260812 19:01:32.091233 14554 ts_tablet_manager.cc:1905] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:01:32.094554 14554 ts_tablet_manager.cc:1918] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:01:32.094740 14554 log.cc:1196] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/9406c4d78dbf4ecd8ad1eab00f30b87c
I20260812 19:01:32.095162 14554 ts_tablet_manager.cc:1939] T 9406c4d78dbf4ecd8ad1eab00f30b87c P a91f7679cd6a4d71949e6eace4c5c66e: Deleting consensus metadata
I20260812 19:01:32.096498 13946 catalog_manager.cc:4841] TS a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635): tablet 9406c4d78dbf4ecd8ad1eab00f30b87c (table test-table [id=ef377b3db56d42a28832fa245f0fa526]) successfully deleted
I20260812 19:01:32.098803 13944 catalog_manager.cc:4841] TS e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655): tablet 9406c4d78dbf4ecd8ad1eab00f30b87c (table test-table [id=ef377b3db56d42a28832fa245f0fa526]) successfully deleted
W20260812 19:01:32.099318 13947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:40085 (error 108)
I20260812 19:01:32.100163 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 14198
I20260812 19:01:32.100467 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.100108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:40085 (local address 127.0.0.1:33966)
0812 19:01:32.100311 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:32.100377 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260812 19:01:32.102798 13947 catalog_manager.cc:4584] TS f6dd4bcff003483ba673140f1aedcf08 (127.29.112.193:40085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9406c4d78dbf4ecd8ad1eab00f30b87c: Network error: Client connection negotiation failed: client connection to 127.29.112.193:40085: connect: Connection refused (error 111)
I20260812 19:01:32.117336 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 14351
I20260812 19:01:32.124291 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 13927
I20260812 19:01:32.133349 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38735
--webserver_interface=127.29.112.254
--webserver_port=45293
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:38735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:32.238692 14557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:32.238965 14557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:32.239003 14557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:32.242923 14557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:01:32.242971 14557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:32.242990 14557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:32.243005 14557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:01:32.243062 14557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:01:32.248016 14557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:38735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=45293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:32.249047 14557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:32.250073 14557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:32.264273 14563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:32.264235 14564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:32.264259 14566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:32.264561 14557 server_base.cc:1034] running on GCE node
I20260812 19:01:32.265095 14557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:32.266142 14557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:32.267292 14557 hybrid_clock.cc:648] HybridClock initialized: now 1786561292267278 us; error 25 us; skew 500 ppm
Aug 12 19:01:32 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561292, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:32.270093 14557 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:01:32.270988 14557 webserver.cc:460] Webserver started at http://127.29.112.254:45293/ using document root <none> and password file <none>
I20260812 19:01:32.271530 14557 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:32.271602 14557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:32.275662 14557 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 19:01:32.277622 14574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:32.278797 14557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:01:32.278982 14557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "e57e976ec05445748ab9d223268b7f7c"
format_stamp: "Formatted at 2026-08-12 19:01:30 on dist-test-slave-6nmv"
server_key: "9437ddf1f0fc1a41cc4e693ceaa51462"
server_key_iv: "173563929a9450537230a81152c9a56d"
server_key_version: "encryptionkey@0"
I20260812 19:01:32.279562 14557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:32.291570 14557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:32.294591 14557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:32.294814 14557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:32.302340 14557 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:38735
I20260812 19:01:32.302696 14638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:38735 every 8 connection(s)
I20260812 19:01:32.303552 14557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:01:32.307909 14639 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:01:32.309675 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 14557
I20260812 19:01:32.309906 14639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Bootstrap starting.
I20260812 19:01:32.310190 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:40085
--local_ip_for_outbound_sockets=127.29.112.193
--tserver_master_addrs=127.29.112.254:38735
--webserver_port=41983
--webserver_interface=127.29.112.193
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:01:32.327543 14639 log.cc:824] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:32.336009 14639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:01:32.336374 14639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Bootstrap complete.
I20260812 19:01:32.340313 14639 raft_consensus.cc:348] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:32.340726 14639 raft_consensus.cc:729] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e57e976ec05445748ab9d223268b7f7c, State: Initialized, Role: FOLLOWER
I20260812 19:01:32.341190 14639 consensus_queue.cc:260] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:32.341338 14639 raft_consensus.cc:388] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:01:32.341436 14639 raft_consensus.cc:482] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:01:32.341557 14639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:01:32.343381 14639 raft_consensus.cc:504] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:32.343703 14639 leader_election.cc:302] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e57e976ec05445748ab9d223268b7f7c; no voters:
I20260812 19:01:32.344029 14639 leader_election.cc:288] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 19:01:32.344135 14645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:01:32.344322 14645 raft_consensus.cc:686] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 2 LEADER]: Becoming Leader. State: Replica: e57e976ec05445748ab9d223268b7f7c, State: Running, Role: LEADER
I20260812 19:01:32.344631 14645 consensus_queue.cc:237] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:32.345278 14639 sys_catalog.cc:564] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:01:32.345639 14645 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e57e976ec05445748ab9d223268b7f7c. Latest consensus state: current_term: 2 leader_uuid: "e57e976ec05445748ab9d223268b7f7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } } }
I20260812 19:01:32.345749 14645 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:32.346216 14646 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e57e976ec05445748ab9d223268b7f7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } } }
I20260812 19:01:32.346374 14646 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:32.346378 14650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:01:32.351208 14650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:01:32.351876 14650 catalog_manager.cc:1259] Loaded cluster ID: c5bd8e85d8984411a18c796a95672c4f
I20260812 19:01:32.351939 14650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:01:32.353312 14650 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 19:01:32.358119 14650 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 19:01:32.363004 14650 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:01:32.363698 14650 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Loaded TSK: 0
I20260812 19:01:32.364784 14650 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 19:01:32.421650 14642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:32.421974 14642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:32.422062 14642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:32.426106 14642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:32.426186 14642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:32.426282 14642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:01:32.431249 14642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:40085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=41983
--tserver_master_addrs=127.29.112.254:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:32.432237 14642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:32.433202 14642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:32.443322 14671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:32.443378 14672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:32.443549 14642 server_base.cc:1034] running on GCE node
W20260812 19:01:32.443773 14674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:32.444208 14642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:32.444852 14642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:32.446027 14642 hybrid_clock.cc:648] HybridClock initialized: now 1786561292446001 us; error 41 us; skew 500 ppm
Aug 12 19:01:32 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561292, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:32.448871 14642 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:01:32.449810 14642 webserver.cc:460] Webserver started at http://127.29.112.193:41983/ using document root <none> and password file <none>
I20260812 19:01:32.450407 14642 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:32.450481 14642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:32.454219 14642 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 19:01:32.456326 14682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:32.457535 14642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:01:32.457620 14642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "f6dd4bcff003483ba673140f1aedcf08"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "2e55d1ad4d266d0e69e344aa45600e4f"
server_key_iv: "4a6afa5abb686c2a06e297b55ddea661"
server_key_version: "encryptionkey@0"
I20260812 19:01:32.458199 14642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:32.472291 14642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:32.475565 14642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:32.475780 14642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:32.476404 14642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:32.477524 14691 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 19:01:32.481638 14642 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 19:01:32.481698 14642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260812 19:01:32.481760 14642 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 19:01:32.483420 14642 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 19:01:32.483474 14642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 19:01:32.483750 14691 tablet_bootstrap.cc:492] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Bootstrap starting.
I20260812 19:01:32.494416 14642 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:40085
I20260812 19:01:32.494436 14812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:40085 every 8 connection(s)
I20260812 19:01:32.495548 14642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:01:32.497284 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 14642
I20260812 19:01:32.497725 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:37635
--local_ip_for_outbound_sockets=127.29.112.194
--tserver_master_addrs=127.29.112.254:38735
--webserver_port=46337
--webserver_interface=127.29.112.194
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:01:32.505242 14691 log.cc:824] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:01:32 dist-test-slave-6nmv krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561292, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:32.512367 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.497580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:41831 (local address 127.29.112.254:38735)
0812 19:01:32.498201 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:32.498222 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:32.498735 (+ 513us) server_negotiation.cc:407] Connection header received
0812 19:01:32.500224 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:32.500234 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:32.500501 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:32.500829 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:32.502008 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.502941 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.504169 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.504628 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.504797 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.507435 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:32.507440 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:32.507451 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:32.507479 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:32.509262 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:32.509801 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:32.509841 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:32.509848 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:32.509984 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:32.510343 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:32.510346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:32.510347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:32.510657 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:32.510792 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:32.511268 (+ 476us) server_negotiation.cc:299] Negotiation successful
0812 19:01:32.511510 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":119,"threads_started":1}
I20260812 19:01:32.512379 14817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.497582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:41831)
0812 19:01:32.498533 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:32.498590 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:32.499794 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:32.500887 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:32.500894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:32.501303 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:32.501765 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.501778 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.503096 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.503105 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.504044 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.504054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.504769 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.504773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.504965 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.505585 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:32.505601 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:32.507244 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:32.509387 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:32.509393 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:32.509405 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:32.509670 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:32.510096 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:32.510101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:32.510102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:32.510248 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:32.510766 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:32.510772 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:32.510993 (+ 221us) client_negotiation.cc:769] Sending connection context
0812 19:01:32.511150 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 19:01:32.511461 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":162,"threads_started":1}
I20260812 19:01:32.514609 14691 tablet_bootstrap.cc:492] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:01:32.514803 14813 heartbeater.cc:347] Connected to a master server at 127.29.112.254:38735
I20260812 19:01:32.515025 14813 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:32.515023 14691 tablet_bootstrap.cc:492] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Bootstrap complete.
I20260812 19:01:32.515569 14813 heartbeater.cc:511] Master 127.29.112.254:38735 requested a full tablet report, sending...
I20260812 19:01:32.516530 14691 ts_tablet_manager.cc:1397] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Time spent bootstrapping tablet: real 0.033s user 0.020s sys 0.010s
I20260812 19:01:32.517786 14593 ts_manager.cc:194] Registered new tserver with Master: f6dd4bcff003483ba673140f1aedcf08 (127.29.112.193:40085)
I20260812 19:01:32.519322 14691 raft_consensus.cc:348] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } }
I20260812 19:01:32.519829 14691 raft_consensus.cc:729] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6dd4bcff003483ba673140f1aedcf08, State: Initialized, Role: FOLLOWER
I20260812 19:01:32.520287 14691 consensus_queue.cc:260] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } }
I20260812 19:01:32.521225 14593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:41831
I20260812 19:01:32.521319 14691 ts_tablet_manager.cc:1428] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 19:01:32.523554 14813 heartbeater.cc:503] Master 127.29.112.254:38735 was elected leader, sending a full tablet report...
I20260812 19:01:32.526726 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.520522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:40085 (local address 127.0.0.1:33982)
0812 19:01:32.521873 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:32.521921 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:32.522053 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:32.522954 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:32.522959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:32.522983 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:32.523099 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.523105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.524291 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.524297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.525321 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.525330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.526186 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.526190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.526310 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.526350 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 19:01:32.526474 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 19:01:32.526596 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1155,"thread_start_us":110,"threads_started":1}
I20260812 19:01:32.526726 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.521241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.29.112.193:40085)
0812 19:01:32.522316 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:32.522326 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:32.522359 (+ 33us) server_negotiation.cc:407] Connection header received
0812 19:01:32.522445 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:32.522449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:32.522777 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:32.522933 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:32.523215 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.524189 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.525442 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.526059 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.526193 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.526322 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:32.526512 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 19:01:32.526595 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":986,"thread_start_us":126,"threads_started":1}
I20260812 19:01:32.527648 14741 tablet_service.cc:1514] Processing DeleteTablet for tablet 9406c4d78dbf4ecd8ad1eab00f30b87c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:32 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:33982
I20260812 19:01:32.528368 14824 tablet_replica.cc:330] stopping tablet replica
I20260812 19:01:32.528641 14824 raft_consensus.cc:2229] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:01:32.528867 14824 raft_consensus.cc:2258] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:01:32.529772 14824 ts_tablet_manager.cc:1905] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:01:32.533118 14824 ts_tablet_manager.cc:1918] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:01:32.533210 14824 log.cc:1196] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal/wals/9406c4d78dbf4ecd8ad1eab00f30b87c
I20260812 19:01:32.533519 14824 ts_tablet_manager.cc:1939] T 9406c4d78dbf4ecd8ad1eab00f30b87c P f6dd4bcff003483ba673140f1aedcf08: Deleting consensus metadata
I20260812 19:01:32.535064 14579 catalog_manager.cc:4841] TS f6dd4bcff003483ba673140f1aedcf08 (127.29.112.193:40085): tablet 9406c4d78dbf4ecd8ad1eab00f30b87c (table test-table [id=ef377b3db56d42a28832fa245f0fa526]) successfully deleted
W20260812 19:01:32.620456 14818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:32.620730 14818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:32.620774 14818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:32.624454 14818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:32.624507 14818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:32.624576 14818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:01:32.629422 14818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:37635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=46337
--tserver_master_addrs=127.29.112.254:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:32.630465 14818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:32.631361 14818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:32.640601 14830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:32.640564 14832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:32.640539 14829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:32.640967 14818 server_base.cc:1034] running on GCE node
I20260812 19:01:32.641367 14818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:32.642174 14818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:32.643368 14818 hybrid_clock.cc:648] HybridClock initialized: now 1786561292643350 us; error 34 us; skew 500 ppm
Aug 12 19:01:32 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561292, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:32.646186 14818 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:01:32.647073 14818 webserver.cc:460] Webserver started at http://127.29.112.194:46337/ using document root <none> and password file <none>
I20260812 19:01:32.647605 14818 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:32.647720 14818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:32.651479 14818 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 19:01:32.653522 14841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:32.654716 14818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 19:01:32.654846 14818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "a91f7679cd6a4d71949e6eace4c5c66e"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "70855a7eae6ca3513133ab21246542ed"
server_key_iv: "13ac2983a1784b0c8e8b2390eb9932cc"
server_key_version: "encryptionkey@0"
I20260812 19:01:32.655372 14818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:32.672569 14818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:32.675937 14818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:32.676204 14818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:32.676836 14818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:32.677696 14818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:32.677767 14818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:32.677871 14818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:32.677925 14818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:32.687178 14818 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:37635
I20260812 19:01:32.687202 14973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:37635 every 8 connection(s)
I20260812 19:01:32.688159 14818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
Aug 12 19:01:32 dist-test-slave-6nmv krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561292, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:32.698395 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 14818
I20260812 19:01:32.698798 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:39655
--local_ip_for_outbound_sockets=127.29.112.195
--tserver_master_addrs=127.29.112.254:38735
--webserver_port=39275
--webserver_interface=127.29.112.195
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:01:32.701385 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.689943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:46187 (local address 127.29.112.254:38735)
0812 19:01:32.690110 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:32.690115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:32.690961 (+ 846us) server_negotiation.cc:407] Connection header received
0812 19:01:32.692267 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:32.692269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:32.692320 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:32.692397 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:32.693542 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.694110 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.695177 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.695391 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.695474 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.697731 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:32.697734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:32.697736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:32.697759 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:32.699381 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:32.699906 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:32.699914 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:32.699915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:32.699992 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:32.700356 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:32.700361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:32.700362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:32.700537 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:32.700638 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:32.701080 (+ 442us) server_negotiation.cc:299] Negotiation successful
0812 19:01:32.701205 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:01:32.702188 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.690389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:46187)
0812 19:01:32.690811 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:32.690851 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:32.692072 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:32.692553 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:32.692559 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:32.692920 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:32.693375 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.693387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.694226 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.694228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.695072 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.695081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.695510 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.695513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.695641 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.696165 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:32.696179 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:32.697583 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:32.699509 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:32.699514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:32.699527 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:32.699787 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:32.700085 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:32.700089 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:32.700091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:32.700247 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:32.700637 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:32.700643 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:32.700860 (+ 217us) client_negotiation.cc:769] Sending connection context
0812 19:01:32.701041 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 19:01:32.701347 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":118,"threads_started":1}
I20260812 19:01:32.703275 14974 heartbeater.cc:347] Connected to a master server at 127.29.112.254:38735
I20260812 19:01:32.703510 14974 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:32.704031 14974 heartbeater.cc:511] Master 127.29.112.254:38735 requested a full tablet report, sending...
I20260812 19:01:32.705035 14593 ts_manager.cc:194] Registered new tserver with Master: a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635)
I20260812 19:01:32.705907 14593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:46187
W20260812 19:01:32.808687 14977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:32.808959 14977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:32.809000 14977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:32.812994 14977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:32.813056 14977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:32.813112 14977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:01:32.817979 14977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:39655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=39275
--tserver_master_addrs=127.29.112.254:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:32.818900 14977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:32.819850 14977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:32.829291 14986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:32.829291 14984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:32.829326 14983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:32.829998 14977 server_base.cc:1034] running on GCE node
I20260812 19:01:32.830467 14977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:32.831215 14977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:32.832376 14977 hybrid_clock.cc:648] HybridClock initialized: now 1786561292832356 us; error 30 us; skew 500 ppm
Aug 12 19:01:32 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561292, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:32.835129 14977 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:01:32.835964 14977 webserver.cc:460] Webserver started at http://127.29.112.195:39275/ using document root <none> and password file <none>
I20260812 19:01:32.836519 14977 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:32.836647 14977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:32.840384 14977 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:01:32.842350 14996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:32.843474 14977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:01:32.843618 14977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "e202ab34ffce4c42a67936b923e37188"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "ac0d19662f4bb47daa93a54efa19d9ad"
server_key_iv: "6e8f01cb85814f4e4323397b97e60269"
server_key_version: "encryptionkey@0"
I20260812 19:01:32.844131 14977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:32.867425 14977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:32.870272 14977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:32.870477 14977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:32.871071 14977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:32.871925 14977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:32.872005 14977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:32.872077 14977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:32.872128 14977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:32.880869 14977 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:39655
I20260812 19:01:32.880894 15130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:39655 every 8 connection(s)
I20260812 19:01:32.881752 14977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:01:32.886073 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 14977
Aug 12 19:01:32 dist-test-slave-6nmv krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561292, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:32.895365 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.886796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59938 (local address 127.29.112.254:38735)
0812 19:01:32.887007 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:32.887011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:32.887132 (+ 121us) server_negotiation.cc:407] Connection header received
0812 19:01:32.887219 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:32.887221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:32.887268 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:32.887361 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:32.888172 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.888904 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.889887 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.890199 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.890295 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.891689 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:32.891692 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:32.891694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:32.891716 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:32.893509 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:32.894039 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:32.894043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:32.894046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:32.894130 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:32.894376 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:32.894379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:32.894381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:32.894605 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:32.894748 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:32.894961 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 19:01:32.895096 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":68,"threads_started":1}
I20260812 19:01:32.896510 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.883673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:41875 (local address 127.29.112.254:38735)
0812 19:01:32.883808 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:32.883813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:32.884639 (+ 826us) server_negotiation.cc:407] Connection header received
0812 19:01:32.885898 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:32.885902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:32.885955 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:32.886023 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:32.887491 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.888228 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.889570 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.889939 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.890018 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.892880 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:32.892885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:32.892889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:32.892929 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:32.894477 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:32.895015 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:32.895018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:32.895019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:32.895091 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:32.895424 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:32.895428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:32.895430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:32.895607 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:32.895697 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:32.896295 (+ 598us) server_negotiation.cc:299] Negotiation successful
0812 19:01:32.896403 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:01:32.897425 15134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.884093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:41875)
0812 19:01:32.884499 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:32.884538 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:32.885660 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:32.886224 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:32.886232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:32.886654 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:32.887331 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.887351 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.888336 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.888339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.889446 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.889455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.890072 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.890075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.890279 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.891043 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:32.891058 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:32.892689 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:32.894576 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:32.894580 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:32.894589 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:32.894929 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:32.895189 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:32.895191 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:32.895193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:32.895329 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:32.895709 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:32.895715 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:32.896022 (+ 307us) client_negotiation.cc:769] Sending connection context
0812 19:01:32.896219 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 19:01:32.896541 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":112,"threads_started":1}
I20260812 19:01:32.898397 15131 heartbeater.cc:347] Connected to a master server at 127.29.112.254:38735
I20260812 19:01:32.898597 15131 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:32.899031 15131 heartbeater.cc:511] Master 127.29.112.254:38735 requested a full tablet report, sending...
I20260812 19:01:32.899924 14593 ts_manager.cc:194] Registered new tserver with Master: e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655)
I20260812 19:01:32.900449 14593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:41875
I20260812 19:01:32.908526 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:01:32.913046 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.909226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59944 (local address 127.29.112.254:38735)
0812 19:01:32.909368 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:32.909374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:32.909576 (+ 202us) server_negotiation.cc:407] Connection header received
0812 19:01:32.909756 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:32.909760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:32.909944 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:32.910059 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:01:32.910333 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.910815 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.911668 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.911903 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.912034 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.912342 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:01:32.912675 (+ 333us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:01:32.912794 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:32.912870 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 19:01:32.912915 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:01:32.913616 14593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:01:32.916596 14593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:01:32.924075 14741 tablet_service.cc:1467] Processing CreateTablet for tablet dfedadd810c94d62b2f1d1b91d625045 (DEFAULT_TABLE table=test-table [id=283841f097464bd8a567b73b5e960d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:32.924715 14741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfedadd810c94d62b2f1d1b91d625045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:32.927456 14691 tablet_bootstrap.cc:492] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: Bootstrap starting.
I20260812 19:01:32.927693 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.922974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39655 (local address 127.0.0.1:44016)
0812 19:01:32.923112 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:32.923126 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:32.923250 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:32.924408 (+ 1158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:32.924411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:32.924429 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:32.924527 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.924532 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.925680 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.925682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.926600 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.926608 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.927347 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.927350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.927421 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.927440 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 19:01:32.927494 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 19:01:32.927560 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":1}
I20260812 19:01:32.927719 15142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.923455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44016 (local address 127.29.112.195:39655)
0812 19:01:32.923920 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:32.923932 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:32.923955 (+ 23us) server_negotiation.cc:407] Connection header received
0812 19:01:32.924025 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:32.924030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:32.924236 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:32.924436 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:32.924627 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.925559 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.926713 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.927232 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.927349 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.927484 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:32.927529 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 19:01:32.927608 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":155,"threads_started":1}
I20260812 19:01:32.928241 14691 tablet_bootstrap.cc:654] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:32.928655 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.923504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:37635 (local address 127.0.0.1:34540)
0812 19:01:32.923763 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:32.923777 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:32.923860 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:32.924695 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:32.924697 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:32.924712 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:32.924793 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.924798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.926294 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.926297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.927323 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:32.927332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.928277 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.928280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:32.928375 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.928396 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:01:32.928443 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 19:01:32.928527 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":130,"threads_started":1}
I20260812 19:01:32.929386 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:32.923966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34540 (local address 127.29.112.194:37635)
0812 19:01:32.924267 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:32.924279 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:32.924303 (+ 24us) server_negotiation.cc:407] Connection header received
0812 19:01:32.924372 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:32.924379 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:32.924577 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:32.924715 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:32.924868 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.926150 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.927463 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:32.928144 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:32.928979 (+ 835us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:32.929096 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:32.929144 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 19:01:32.929244 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":115,"threads_started":1}
I20260812 19:01:32.929540 14691 tablet_bootstrap.cc:492] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: No bootstrap required, opened a new log
I20260812 19:01:32.929631 14691 ts_tablet_manager.cc:1397] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 19:01:32.930295 14691 raft_consensus.cc:348] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:32.930403 14691 raft_consensus.cc:374] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:32.930442 14691 raft_consensus.cc:729] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6dd4bcff003483ba673140f1aedcf08, State: Initialized, Role: FOLLOWER
I20260812 19:01:32.930562 14691 consensus_queue.cc:260] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:32.930821 14691 ts_tablet_manager.cc:1428] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 19:01:32.930852 15059 tablet_service.cc:1467] Processing CreateTablet for tablet dfedadd810c94d62b2f1d1b91d625045 (DEFAULT_TABLE table=test-table [id=283841f097464bd8a567b73b5e960d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:32.931953 15059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfedadd810c94d62b2f1d1b91d625045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:32.932052 14903 tablet_service.cc:1467] Processing CreateTablet for tablet dfedadd810c94d62b2f1d1b91d625045 (DEFAULT_TABLE table=test-table [id=283841f097464bd8a567b73b5e960d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:32.933041 14903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfedadd810c94d62b2f1d1b91d625045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:32.939756 15146 tablet_bootstrap.cc:492] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188: Bootstrap starting.
I20260812 19:01:32.940451 15147 tablet_bootstrap.cc:492] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e: Bootstrap starting.
I20260812 19:01:32.942353 15146 tablet_bootstrap.cc:654] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:32.942737 15147 tablet_bootstrap.cc:654] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:32.943086 15146 log.cc:824] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:32.943426 15147 log.cc:824] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:32.944783 15146 tablet_bootstrap.cc:492] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188: No bootstrap required, opened a new log
I20260812 19:01:32.944978 15146 ts_tablet_manager.cc:1397] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 19:01:32.948153 15146 raft_consensus.cc:348] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:32.948374 15146 raft_consensus.cc:374] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:32.948441 15146 raft_consensus.cc:729] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e202ab34ffce4c42a67936b923e37188, State: Initialized, Role: FOLLOWER
I20260812 19:01:32.948854 15146 consensus_queue.cc:260] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:32.949313 15147 tablet_bootstrap.cc:492] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e: No bootstrap required, opened a new log
I20260812 19:01:32.949512 15147 ts_tablet_manager.cc:1397] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260812 19:01:32.949522 15131 heartbeater.cc:503] Master 127.29.112.254:38735 was elected leader, sending a full tablet report...
I20260812 19:01:32.950057 15146 ts_tablet_manager.cc:1428] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260812 19:01:32.953169 15147 raft_consensus.cc:348] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:32.953382 15147 raft_consensus.cc:374] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:32.953445 15147 raft_consensus.cc:729] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a91f7679cd6a4d71949e6eace4c5c66e, State: Initialized, Role: FOLLOWER
I20260812 19:01:32.953801 15147 consensus_queue.cc:260] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:32.954468 14974 heartbeater.cc:503] Master 127.29.112.254:38735 was elected leader, sending a full tablet report...
I20260812 19:01:32.954720 15147 ts_tablet_manager.cc:1428] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
W20260812 19:01:32.996902 14814 tablet.cc:2369] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:01:33.029218 14820 raft_consensus.cc:482] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:01:33.029484 14820 raft_consensus.cc:504] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:33.030606 14820 leader_election.cc:288] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655), a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635)
I20260812 19:01:33.035879 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.030986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:54373 (local address 127.29.112.194:37635)
0812 19:01:33.031176 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:33.031181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:33.031361 (+ 180us) server_negotiation.cc:407] Connection header received
0812 19:01:33.031456 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:33.031459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:33.031509 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:33.031590 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:33.031974 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:33.032756 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:33.034612 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:33.035187 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:33.035301 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:33.035348 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:33.035695 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 19:01:33.035758 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 19:01:33.035879 15142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.031055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:43161 (local address 127.29.112.195:39655)
0812 19:01:33.031189 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:33.031195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:33.031299 (+ 104us) server_negotiation.cc:407] Connection header received
0812 19:01:33.031450 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:33.031454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:33.031505 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:33.031589 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:33.031981 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:33.032750 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:33.034613 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:33.035187 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:33.035301 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:33.035348 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:33.035695 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 19:01:33.035759 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 19:01:33.035924 15156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.030836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:37635 (local address 127.29.112.193:54373)
0812 19:01:33.031265 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.031290 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:33.031363 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:33.031632 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:33.031635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:33.031683 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:33.031848 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:33.031856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:33.032867 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:33.032870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:33.034492 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:33.034498 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:33.035305 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:33.035308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:33.035553 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:33.035594 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 19:01:33.035667 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 19:01:33.035760 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":407,"spinlock_wait_cycles":768,"thread_start_us":98,"threads_started":1}
I20260812 19:01:33.035924 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.030836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39655 (local address 127.29.112.193:43161)
0812 19:01:33.031187 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.031203 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:33.031338 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:33.031632 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:33.031635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:33.031699 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:33.031861 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:33.031866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:33.032867 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:33.032870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:33.034485 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:33.034496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:33.035305 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:33.035308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:33.035548 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:33.035592 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 19:01:33.035667 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 19:01:33.035760 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":264,"spinlock_wait_cycles":96640,"thread_start_us":70,"threads_started":1}
I20260812 19:01:33.036577 14925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfedadd810c94d62b2f1d1b91d625045" candidate_uuid: "f6dd4bcff003483ba673140f1aedcf08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" is_pre_election: true
I20260812 19:01:33.036626 15080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfedadd810c94d62b2f1d1b91d625045" candidate_uuid: "f6dd4bcff003483ba673140f1aedcf08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e202ab34ffce4c42a67936b923e37188" is_pre_election: true
I20260812 19:01:33.036916 15080 raft_consensus.cc:2454] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6dd4bcff003483ba673140f1aedcf08 in term 0.
I20260812 19:01:33.036916 14925 raft_consensus.cc:2454] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6dd4bcff003483ba673140f1aedcf08 in term 0.
I20260812 19:01:33.037443 14685 leader_election.cc:302] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a91f7679cd6a4d71949e6eace4c5c66e, e202ab34ffce4c42a67936b923e37188, f6dd4bcff003483ba673140f1aedcf08; no voters:
I20260812 19:01:33.037720 14820 raft_consensus.cc:2781] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:01:33.037823 14820 raft_consensus.cc:482] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:01:33.037880 14820 raft_consensus.cc:3037] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:33.038946 14820 raft_consensus.cc:504] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:33.039288 14820 leader_election.cc:288] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 1 election: Requested vote from peers e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655), a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635)
I20260812 19:01:33.039639 15080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfedadd810c94d62b2f1d1b91d625045" candidate_uuid: "f6dd4bcff003483ba673140f1aedcf08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e202ab34ffce4c42a67936b923e37188"
I20260812 19:01:33.039633 14925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfedadd810c94d62b2f1d1b91d625045" candidate_uuid: "f6dd4bcff003483ba673140f1aedcf08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a91f7679cd6a4d71949e6eace4c5c66e"
I20260812 19:01:33.039757 14925 raft_consensus.cc:3037] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:33.039757 15080 raft_consensus.cc:3037] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:33.041047 14925 raft_consensus.cc:2454] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6dd4bcff003483ba673140f1aedcf08 in term 1.
I20260812 19:01:33.041047 15080 raft_consensus.cc:2454] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6dd4bcff003483ba673140f1aedcf08 in term 1.
I20260812 19:01:33.041311 14685 leader_election.cc:302] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a91f7679cd6a4d71949e6eace4c5c66e, f6dd4bcff003483ba673140f1aedcf08; no voters:
I20260812 19:01:33.041510 14820 raft_consensus.cc:2781] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:01:33.041712 14820 raft_consensus.cc:686] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 LEADER]: Becoming Leader. State: Replica: f6dd4bcff003483ba673140f1aedcf08, State: Running, Role: LEADER
I20260812 19:01:33.041975 14820 consensus_queue.cc:237] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:33.044654 14593 catalog_manager.cc:5495] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6dd4bcff003483ba673140f1aedcf08 (127.29.112.193). New cstate: current_term: 1 leader_uuid: "f6dd4bcff003483ba673140f1aedcf08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } health_report { overall_health: HEALTHY } } }
I20260812 19:01:33.083072 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.079532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33998 (local address 127.29.112.193:40085)
0812 19:01:33.079815 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:33.079822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:33.079836 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:01:33.079910 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:33.079912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:33.079966 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:33.080068 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:01:33.080294 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:33.080829 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:33.081611 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:33.081895 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:33.081975 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:33.082238 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:01:33.082717 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:01:33.082809 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:33.082892 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 19:01:33.082942 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":119,"threads_started":1}
I20260812 19:01:33.087196 15080 raft_consensus.cc:1261] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188 [term 1 FOLLOWER]: Refusing update from remote peer f6dd4bcff003483ba673140f1aedcf08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:01:33.087196 14925 raft_consensus.cc:1261] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e [term 1 FOLLOWER]: Refusing update from remote peer f6dd4bcff003483ba673140f1aedcf08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:01:33.087932 15158 consensus_queue.cc:1035] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:01:33.088189 14820 consensus_queue.cc:1035] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:01:33.096879 15163 mvcc.cc:205] Tried to move back new op lower bound from 7317755056478273536 to 7317755056302211072. Current Snapshot: MvccSnapshot[applied={T|T < 7317755056478273536}]
I20260812 19:01:33.104548 14592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:01:33.104764 14592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:01:33.107281 14592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Sending DeleteTablet for 3 replicas of tablet dfedadd810c94d62b2f1d1b91d625045
I20260812 19:01:33.108078 14741 tablet_service.cc:1514] Processing DeleteTablet for tablet dfedadd810c94d62b2f1d1b91d625045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:33 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:33982
I20260812 19:01:33.108125 15059 tablet_service.cc:1514] Processing DeleteTablet for tablet dfedadd810c94d62b2f1d1b91d625045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:33 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:44016
I20260812 19:01:33.108121 14903 tablet_service.cc:1514] Processing DeleteTablet for tablet dfedadd810c94d62b2f1d1b91d625045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:33 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:34540
I20260812 19:01:33.108779 15173 tablet_replica.cc:330] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: stopping tablet replica
I20260812 19:01:33.108925 15173 raft_consensus.cc:2229] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:01:33.109143 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 14642
I20260812 19:01:33.109268 15173 raft_consensus.cc:2258] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:01:33.112093 15175 tablet_replica.cc:330] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e: stopping tablet replica
I20260812 19:01:33.112411 15175 raft_consensus.cc:2229] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:01:33.112703 15175 raft_consensus.cc:2258] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:01:33.113054 15174 tablet_replica.cc:330] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188: stopping tablet replica
I20260812 19:01:33.113349 15174 raft_consensus.cc:2229] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:01:33.113591 15174 raft_consensus.cc:2258] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:01:33.115185 15175 ts_tablet_manager.cc:1905] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:01:33.115865 15174 ts_tablet_manager.cc:1905] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:01:33.117549 15175 ts_tablet_manager.cc:1918] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:01:33.117841 15175 log.cc:1196] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/dfedadd810c94d62b2f1d1b91d625045
I20260812 19:01:33.118237 15175 ts_tablet_manager.cc:1939] T dfedadd810c94d62b2f1d1b91d625045 P a91f7679cd6a4d71949e6eace4c5c66e: Deleting consensus metadata
I20260812 19:01:33.119473 14578 catalog_manager.cc:4841] TS a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635): tablet dfedadd810c94d62b2f1d1b91d625045 (table test-table [id=283841f097464bd8a567b73b5e960d8b]) successfully deleted
I20260812 19:01:33.119993 15174 ts_tablet_manager.cc:1918] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:01:33.120144 15174 log.cc:1196] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/dfedadd810c94d62b2f1d1b91d625045
I20260812 19:01:33.120667 15174 ts_tablet_manager.cc:1939] T dfedadd810c94d62b2f1d1b91d625045 P e202ab34ffce4c42a67936b923e37188: Deleting consensus metadata
I20260812 19:01:33.122305 14575 catalog_manager.cc:4841] TS e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655): tablet dfedadd810c94d62b2f1d1b91d625045 (table test-table [id=283841f097464bd8a567b73b5e960d8b]) successfully deleted
W20260812 19:01:33.130705 14579 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:40085 (error 108)
I20260812 19:01:33.131680 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 14818
I20260812 19:01:33.131882 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.131522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:40085 (local address 127.0.0.1:34004)
0812 19:01:33.131685 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.131774 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:40085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 19:01:33.136039 14579 catalog_manager.cc:4584] TS f6dd4bcff003483ba673140f1aedcf08 (127.29.112.193:40085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dfedadd810c94d62b2f1d1b91d625045: Network error: Client connection negotiation failed: client connection to 127.29.112.193:40085: connect: Connection refused (error 111)
I20260812 19:01:33.140925 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 14977
I20260812 19:01:33.148952 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 14557
I20260812 19:01:33.160096 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38735
--webserver_interface=127.29.112.254
--webserver_port=45293
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:38735
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:33.272837 15177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:33.273133 15177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:33.273181 15177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:33.277292 15177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:01:33.277361 15177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:33.277384 15177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:33.277405 15177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:01:33.277423 15177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:01:33.282840 15177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:38735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=45293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:33.284024 15177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:33.285012 15177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:33.293515 15183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:33.293579 15177 server_base.cc:1034] running on GCE node
W20260812 19:01:33.293483 15182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:33.293483 15185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:33.294291 15177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:33.295408 15177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:33.296613 15177 hybrid_clock.cc:648] HybridClock initialized: now 1786561293296596 us; error 28 us; skew 500 ppm
Aug 12 19:01:33 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561293, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:33.299690 15177 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:01:33.300585 15177 webserver.cc:460] Webserver started at http://127.29.112.254:45293/ using document root <none> and password file <none>
I20260812 19:01:33.301143 15177 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:33.301235 15177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:33.305270 15177 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:01:33.307562 15195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:33.308827 15177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:01:33.308967 15177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "e57e976ec05445748ab9d223268b7f7c"
format_stamp: "Formatted at 2026-08-12 19:01:30 on dist-test-slave-6nmv"
server_key: "9437ddf1f0fc1a41cc4e693ceaa51462"
server_key_iv: "173563929a9450537230a81152c9a56d"
server_key_version: "encryptionkey@0"
I20260812 19:01:33.309518 15177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:33.327122 15177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:33.330179 15177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:33.330372 15177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:33.337342 15177 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:38735
I20260812 19:01:33.337636 15257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:38735 every 8 connection(s)
I20260812 19:01:33.338398 15177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:01:33.342588 15258 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:01:33.344261 15258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Bootstrap starting.
I20260812 19:01:33.347535 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 15177
I20260812 19:01:33.347985 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:40085
--local_ip_for_outbound_sockets=127.29.112.193
--tserver_master_addrs=127.29.112.254:38735
--webserver_port=41983
--webserver_interface=127.29.112.193
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:01:33.365892 15258 log.cc:824] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:33.380185 15258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:01:33.380626 15258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c: Bootstrap complete.
I20260812 19:01:33.386170 15258 raft_consensus.cc:348] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:33.386690 15258 raft_consensus.cc:729] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e57e976ec05445748ab9d223268b7f7c, State: Initialized, Role: FOLLOWER
I20260812 19:01:33.387231 15258 consensus_queue.cc:260] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:33.387362 15258 raft_consensus.cc:388] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:01:33.387458 15258 raft_consensus.cc:482] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:01:33.387566 15258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:01:33.389490 15258 raft_consensus.cc:504] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:33.389851 15258 leader_election.cc:302] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e57e976ec05445748ab9d223268b7f7c; no voters:
I20260812 19:01:33.390228 15258 leader_election.cc:288] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 19:01:33.390324 15263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:01:33.390532 15263 raft_consensus.cc:686] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [term 3 LEADER]: Becoming Leader. State: Replica: e57e976ec05445748ab9d223268b7f7c, State: Running, Role: LEADER
I20260812 19:01:33.390825 15263 consensus_queue.cc:237] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } }
I20260812 19:01:33.391443 15258 sys_catalog.cc:564] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:01:33.391778 15264 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e57e976ec05445748ab9d223268b7f7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } } }
I20260812 19:01:33.391999 15264 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:33.392279 15265 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e57e976ec05445748ab9d223268b7f7c. Latest consensus state: current_term: 3 leader_uuid: "e57e976ec05445748ab9d223268b7f7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57e976ec05445748ab9d223268b7f7c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38735 } } }
I20260812 19:01:33.392508 15265 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57e976ec05445748ab9d223268b7f7c [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:33.393000 15267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:01:33.398550 15267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:01:33.399288 15267 catalog_manager.cc:1259] Loaded cluster ID: c5bd8e85d8984411a18c796a95672c4f
I20260812 19:01:33.399353 15267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 19:01:33.400439 15267 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f52c5bff9ad google::LogMessage::Fail() at ??:0
@ 0x7f52c5c0288a google::LogMessage::SendToLog() at ??:0
@ 0x7f52c5bff4de google::LogMessage::Flush() at ??:0
@ 0x7f52c5c00a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f52d053efdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f52d053fd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f52d053c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f52d0580640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f52c7e513d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f52c7e51c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f52c7e533f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f52c7e42512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f52c94866db start_thread at ??:0
@ 0x7f52c46fd61f clone at ??:0
W20260812 19:01:33.491110 15261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:33.491372 15261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:33.491477 15261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:33.495584 15261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:33.495664 15261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:33.495780 15261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:01:33.500756 15261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:40085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=41983
--tserver_master_addrs=127.29.112.254:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:33.501751 15261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:33.502723 15261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:33.512488 15303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:33.512504 15305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:33.512494 15307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:33.513914 15261 server_base.cc:1034] running on GCE node
I20260812 19:01:33.514420 15261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:33.515224 15261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:33.516439 15261 hybrid_clock.cc:648] HybridClock initialized: now 1786561293516399 us; error 52 us; skew 500 ppm
Aug 12 19:01:33 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561293, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:33.519757 15261 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:01:33.520673 15261 webserver.cc:460] Webserver started at http://127.29.112.193:41983/ using document root <none> and password file <none>
I20260812 19:01:33.521214 15261 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:33.521301 15261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:33.525137 15261 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:01:33.527566 15314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:33.529127 15261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 19:01:33.529259 15261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "f6dd4bcff003483ba673140f1aedcf08"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "2e55d1ad4d266d0e69e344aa45600e4f"
server_key_iv: "4a6afa5abb686c2a06e297b55ddea661"
server_key_version: "encryptionkey@0"
I20260812 19:01:33.529788 15261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:33.558597 15261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:33.562168 15261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:33.562364 15261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:33.562948 15261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:33.564041 15321 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 19:01:33.569581 15261 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 19:01:33.569670 15261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260812 19:01:33.569756 15261 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 19:01:33.571363 15261 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 19:01:33.571439 15261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 19:01:33.571763 15321 tablet_bootstrap.cc:492] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: Bootstrap starting.
I20260812 19:01:33.582194 15261 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:40085
I20260812 19:01:33.583151 15261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:01:33.583926 15451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:40085 every 8 connection(s)
I20260812 19:01:33.586712 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 15261
I20260812 19:01:33.587117 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:37635
--local_ip_for_outbound_sockets=127.29.112.194
--tserver_master_addrs=127.29.112.254:38735
--webserver_port=46337
--webserver_interface=127.29.112.194
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:01:33.594897 15321 log.cc:824] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:33.603698 15321 tablet_bootstrap.cc:492] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:01:33.604099 15321 tablet_bootstrap.cc:492] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: Bootstrap complete.
I20260812 19:01:33.605592 15321 ts_tablet_manager.cc:1397] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: Time spent bootstrapping tablet: real 0.034s user 0.018s sys 0.012s
I20260812 19:01:33.608654 15321 raft_consensus.cc:348] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:33.609153 15321 raft_consensus.cc:729] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6dd4bcff003483ba673140f1aedcf08, State: Initialized, Role: FOLLOWER
I20260812 19:01:33.609653 15321 consensus_queue.cc:260] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:33.610667 15321 ts_tablet_manager.cc:1428] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08: Time spent starting tablet: real 0.005s user 0.004s sys 0.004s
W20260812 19:01:33.644255 15455 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:01:33.585074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:47737)
0812 19:01:33.585521 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.585568 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:33.586324 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:33.643517 (+ 57193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":145,"threads_started":1}
W20260812 19:01:33.644733 15453 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:38735 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.254:38735: Client connection negotiation failed: client connection to 127.29.112.254:38735: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 19:01:33.645475 15455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.645250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:44291)
0812 19:01:33.645363 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.645428 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 19:01:33.646359 15455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.646172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:34295)
0812 19:01:33.646282 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.646320 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 19:01:33.646584 15453 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:01:33.705047 15456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:33.705474 15456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:33.705557 15456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:33.710729 15456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:33.710790 15456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:33.710876 15456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:01:33.716214 15456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:37635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=46337
--tserver_master_addrs=127.29.112.254:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:33.717235 15456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:33.718326 15456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:33.728049 15464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:33.728130 15456 server_base.cc:1034] running on GCE node
W20260812 19:01:33.728077 15468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:33.728060 15466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:33.728664 15456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:33.729270 15456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:33.730463 15456 hybrid_clock.cc:648] HybridClock initialized: now 1786561293730449 us; error 28 us; skew 500 ppm
Aug 12 19:01:33 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561293, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:33.733283 15456 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:01:33.734264 15456 webserver.cc:460] Webserver started at http://127.29.112.194:46337/ using document root <none> and password file <none>
I20260812 19:01:33.734769 15456 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:33.734840 15456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:33.738427 15456 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:01:33.740502 15475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:33.741693 15456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:01:33.741784 15456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "a91f7679cd6a4d71949e6eace4c5c66e"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "70855a7eae6ca3513133ab21246542ed"
server_key_iv: "13ac2983a1784b0c8e8b2390eb9932cc"
server_key_version: "encryptionkey@0"
I20260812 19:01:33.742352 15456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:33.768923 15456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:33.772109 15456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:33.772264 15456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:33.772797 15456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:33.773584 15456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:33.773631 15456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:33.773669 15456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:33.773698 15456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:33.782522 15456 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:37635
I20260812 19:01:33.782547 15612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:37635 every 8 connection(s)
I20260812 19:01:33.783468 15456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:01:33.784133 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 15456
I20260812 19:01:33.784600 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:39655
--local_ip_for_outbound_sockets=127.29.112.195
--tserver_master_addrs=127.29.112.254:38735
--webserver_port=39275
--webserver_interface=127.29.112.195
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:01:33.787653 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.785612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:59261)
0812 19:01:33.786140 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.786303 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":192,"threads_started":1}
W20260812 19:01:33.788105 15613 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:38735 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.254:38735: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
I20260812 19:01:33.788770 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.788474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:60927)
0812 19:01:33.788619 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.788682 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 19:01:33.789505 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.789234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:39289)
0812 19:01:33.789367 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.789430 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 19:01:33.789705 15613 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:01:33.892560 15616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:33.892786 15616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:33.892829 15616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:33.896586 15616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:33.896633 15616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:33.896703 15616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:01:33.901604 15616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:39655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=39275
--tserver_master_addrs=127.29.112.254:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:33.902514 15616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:33.903491 15616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:33.912889 15626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:33.912823 15623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:33.912933 15622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:33.913045 15616 server_base.cc:1034] running on GCE node
I20260812 19:01:33.913555 15616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:33.914199 15616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:33.915411 15616 hybrid_clock.cc:648] HybridClock initialized: now 1786561293915409 us; error 32 us; skew 500 ppm
Aug 12 19:01:33 dist-test-slave-6nmv krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561293, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:33.917946 15616 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:01:33.918782 15616 webserver.cc:460] Webserver started at http://127.29.112.195:39275/ using document root <none> and password file <none>
I20260812 19:01:33.919358 15616 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:33.919449 15616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:33.923167 15616 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 19:01:33.925137 15634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:33.926291 15616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:01:33.926415 15616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "e202ab34ffce4c42a67936b923e37188"
format_stamp: "Formatted at 2026-08-12 19:01:31 on dist-test-slave-6nmv"
server_key: "ac0d19662f4bb47daa93a54efa19d9ad"
server_key_iv: "6e8f01cb85814f4e4323397b97e60269"
server_key_version: "encryptionkey@0"
I20260812 19:01:33.926931 15616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:33.949940 15616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:33.952925 15616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:33.953125 15616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:33.953702 15616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:33.954515 15616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:33.954561 15616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:33.954638 15616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:33.954686 15616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:33.963739 15616 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:39655
I20260812 19:01:33.963766 15771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:39655 every 8 connection(s)
I20260812 19:01:33.964692 15616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:01:33.968477 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.966809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:41703)
0812 19:01:33.967282 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.967392 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":146,"threads_started":1}
W20260812 19:01:33.968940 15772 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:38735 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.254:38735: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
I20260812 19:01:33.969476 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.969275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:60917)
0812 19:01:33.969378 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.969425 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 19:01:33.970124 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:33.969925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:56203)
0812 19:01:33.970030 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:33.970079 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 19:01:33.970302 15772 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 19:01:33.971850 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 15616
I20260812 19:01:34.647585 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:34.647162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:33993)
0812 19:01:34.647404 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:34.647468 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":77,"threads_started":1}
I20260812 19:01:34.790617 15779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:34.790215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:37957)
0812 19:01:34.790469 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:34.790522 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260812 19:01:34.971128 15780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:34.970773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:36421)
0812 19:01:34.970987 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:34.971036 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":69,"threads_started":1}
I20260812 19:01:35.205124 15781 raft_consensus.cc:482] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:01:35.205339 15781 raft_consensus.cc:504] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:35.206499 15781 leader_election.cc:288] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655), a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635)
Aug 12 19:01:35 dist-test-slave-6nmv krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561293, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.194@KRBTEST.COM
Aug 12 19:01:35 dist-test-slave-6nmv krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561293, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.195@KRBTEST.COM
I20260812 19:01:35.221086 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:35.206972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:50553 (local address 127.29.112.195:39655)
0812 19:01:35.207417 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:35.207436 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:35.207478 (+ 42us) server_negotiation.cc:407] Connection header received
0812 19:01:35.208131 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:35.208136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:35.208473 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:35.208769 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:35.210143 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:35.211086 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:35.212383 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:35.212986 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:35.213153 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:35.216403 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:35.216413 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:35.216428 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:35.216463 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:35.218479 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:35.218886 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:35.218894 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:35.218901 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:35.219026 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:35.219316 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:35.219319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:35.219321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:35.219722 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:35.219861 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:35.220630 (+ 769us) server_negotiation.cc:299] Negotiation successful
0812 19:01:35.220902 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":139,"threads_started":1}
I20260812 19:01:35.221088 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:35.207004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:60391 (local address 127.29.112.194:37635)
0812 19:01:35.207385 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:35.207431 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:35.207470 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:01:35.208124 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:35.208130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:35.208473 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:35.208768 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:35.210149 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:35.211088 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:35.212367 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:35.212995 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:35.213169 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:35.216029 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:35.216035 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:35.216050 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:35.216084 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:35.218185 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:35.218696 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:35.218701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:35.218704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:35.218809 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:35.219193 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:35.219196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:35.219197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:35.219722 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:35.219861 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:35.220630 (+ 769us) server_negotiation.cc:299] Negotiation successful
0812 19:01:35.220902 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":103,"threads_started":1}
I20260812 19:01:35.221082 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:35.206812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:37635 (local address 127.29.112.193:60391)
0812 19:01:35.207221 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:35.207236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:35.207327 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:35.208814 (+ 1487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:35.208819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:35.209454 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:35.209973 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:35.209991 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:35.211227 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:35.211230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:35.212231 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:35.212249 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:35.213123 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:35.213127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:35.213300 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:35.214062 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:35.214077 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:35.215773 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:35.218295 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:35.218299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:35.218310 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:35.218586 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:35.218935 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:35.218937 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:35.218938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:35.219059 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:35.219850 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:35.219856 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:35.220254 (+ 398us) client_negotiation.cc:769] Sending connection context
0812 19:01:35.220492 (+ 238us) client_negotiation.cc:240] Negotiation successful
0812 19:01:35.220859 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":272,"spinlock_wait_cycles":32896,"thread_start_us":116,"threads_started":1}
I20260812 19:01:35.221082 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:35.206632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39655 (local address 127.29.112.193:50553)
0812 19:01:35.206961 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:35.206977 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:35.207086 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:35.208813 (+ 1727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:35.208819 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:35.209434 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:35.209974 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:35.209990 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:35.211221 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:35.211230 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:35.212237 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:35.212252 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:35.213112 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:35.213116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:35.213343 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:35.214072 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:35.214096 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:35.216185 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:35.218579 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:35.218581 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:35.218583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:35.218785 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:35.219118 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:35.219122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:35.219124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:35.219229 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:35.219850 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:35.219856 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:35.220254 (+ 398us) client_negotiation.cc:769] Sending connection context
0812 19:01:35.220492 (+ 238us) client_negotiation.cc:240] Negotiation successful
0812 19:01:35.220913 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":369,"spinlock_wait_cycles":6656,"thread_start_us":63,"threads_started":1}
W20260812 19:01:35.224050 15315 leader_election.cc:341] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655): Not found: Tablet not found: dfedadd810c94d62b2f1d1b91d625045
W20260812 19:01:35.224218 15317 leader_election.cc:341] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635): Not found: Tablet not found: dfedadd810c94d62b2f1d1b91d625045
I20260812 19:01:35.224332 15317 leader_election.cc:302] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6dd4bcff003483ba673140f1aedcf08; no voters: a91f7679cd6a4d71949e6eace4c5c66e, e202ab34ffce4c42a67936b923e37188
I20260812 19:01:35.224622 15781 raft_consensus.cc:2726] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:01:35.648743 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:35.648393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:55771)
0812 19:01:35.648586 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:35.648651 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260812 19:01:35.791837 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:35.791417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:48769)
0812 19:01:35.791683 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:35.791746 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":80,"threads_started":1}
I20260812 19:01:35.972229 15787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:35.971849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:60441)
0812 19:01:35.972087 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:35.972141 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":72,"threads_started":1}
I20260812 19:01:36.650084 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:36.649588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:33545)
0812 19:01:36.649910 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:36.649973 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":78,"threads_started":1}
I20260812 19:01:36.793116 15789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:36.792675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:52623)
0812 19:01:36.792925 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:36.792993 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":85,"threads_started":1}
I20260812 19:01:36.973414 15790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:36.972969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:44217)
0812 19:01:36.973241 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:36.973298 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":94,"threads_started":1}
I20260812 19:01:37.203095 15791 raft_consensus.cc:482] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:01:37.203253 15791 raft_consensus.cc:504] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:37.203644 15791 leader_election.cc:288] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655), a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635)
W20260812 19:01:37.204429 15317 leader_election.cc:341] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635): Not found: Tablet not found: dfedadd810c94d62b2f1d1b91d625045
W20260812 19:01:37.204530 15315 leader_election.cc:341] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655): Not found: Tablet not found: dfedadd810c94d62b2f1d1b91d625045
I20260812 19:01:37.204663 15315 leader_election.cc:302] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6dd4bcff003483ba673140f1aedcf08; no voters: a91f7679cd6a4d71949e6eace4c5c66e, e202ab34ffce4c42a67936b923e37188
I20260812 19:01:37.204835 15791 raft_consensus.cc:2726] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:01:37.651180 15792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:37.650817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:47817)
0812 19:01:37.651039 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:37.651089 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260812 19:01:37.794293 15794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:37.793881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:34971)
0812 19:01:37.794152 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:37.794204 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":95,"threads_started":1}
I20260812 19:01:37.974617 15795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:37.974212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:37867)
0812 19:01:37.974469 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:37.974527 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":92,"threads_started":1}
I20260812 19:01:38.652508 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:38.652056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:54569)
0812 19:01:38.652342 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:38.652395 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":90,"threads_started":1}
I20260812 19:01:38.795437 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:38.795029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:39975)
0812 19:01:38.795305 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:38.795354 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":119,"threads_started":1}
I20260812 19:01:38.975667 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:38.975292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:56473)
0812 19:01:38.975536 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:38.975585 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":100,"threads_started":1}
I20260812 19:01:39.447624 15800 raft_consensus.cc:482] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:01:39.447762 15800 raft_consensus.cc:504] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:39.448235 15800 leader_election.cc:288] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655), a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635)
W20260812 19:01:39.449000 15315 leader_election.cc:341] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655): Not found: Tablet not found: dfedadd810c94d62b2f1d1b91d625045
W20260812 19:01:39.449147 15317 leader_election.cc:341] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635): Not found: Tablet not found: dfedadd810c94d62b2f1d1b91d625045
I20260812 19:01:39.449218 15317 leader_election.cc:302] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6dd4bcff003483ba673140f1aedcf08; no voters: a91f7679cd6a4d71949e6eace4c5c66e, e202ab34ffce4c42a67936b923e37188
I20260812 19:01:39.449339 15800 raft_consensus.cc:2726] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:01:39.653541 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:39.653177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:50333)
0812 19:01:39.653397 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:39.653450 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
I20260812 19:01:39.796586 15802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:39.796181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:47481)
0812 19:01:39.796430 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:39.796491 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
I20260812 19:01:39.976773 15803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:39.976375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:59827)
0812 19:01:39.976628 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:39.976683 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
I20260812 19:01:40.655006 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:40.654410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:56017)
0812 19:01:40.654747 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:40.654830 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":128,"threads_started":1}
I20260812 19:01:40.798159 15805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:40.797627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:47189)
0812 19:01:40.798002 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:40.798064 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":96,"threads_started":1}
I20260812 19:01:40.978245 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:40.977698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:41249)
0812 19:01:40.978092 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:40.978155 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":130,"threads_started":1}
I20260812 19:01:41.656479 15807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:41.656000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:52591)
0812 19:01:41.656288 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:41.656366 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":82,"threads_started":1}
I20260812 19:01:41.799551 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:41.799091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:56959)
0812 19:01:41.799375 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:41.799443 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":86,"threads_started":1}
I20260812 19:01:41.979594 15809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:41.979131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:59685)
0812 19:01:41.979423 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:41.979492 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":78,"threads_started":1}
I20260812 19:01:42.385363 15810 raft_consensus.cc:482] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:01:42.385514 15810 raft_consensus.cc:504] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:42.386111 15810 leader_election.cc:288] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655), a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635)
W20260812 19:01:42.386976 15315 leader_election.cc:341] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655): Not found: Tablet not found: dfedadd810c94d62b2f1d1b91d625045
W20260812 19:01:42.387084 15317 leader_election.cc:341] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635): Not found: Tablet not found: dfedadd810c94d62b2f1d1b91d625045
I20260812 19:01:42.387209 15317 leader_election.cc:302] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6dd4bcff003483ba673140f1aedcf08; no voters: a91f7679cd6a4d71949e6eace4c5c66e, e202ab34ffce4c42a67936b923e37188
I20260812 19:01:42.387401 15810 raft_consensus.cc:2726] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:01:42.657723 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:42.657254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:57499)
0812 19:01:42.657567 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:42.657625 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":82,"threads_started":1}
I20260812 19:01:42.800774 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:42.800357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:36049)
0812 19:01:42.800619 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:42.800678 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":82,"threads_started":1}
I20260812 19:01:42.980751 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:42.980337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:51771)
0812 19:01:42.980605 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:42.980663 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":77,"threads_started":1}
I20260812 19:01:43.659068 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:43.658624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:33267)
0812 19:01:43.658905 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:43.658971 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":101,"threads_started":1}
I20260812 19:01:43.802176 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:43.801666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:45805)
0812 19:01:43.802018 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:43.802078 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":78,"threads_started":1}
I20260812 19:01:43.982101 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:43.981592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:36423)
0812 19:01:43.981929 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:43.982000 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":79,"threads_started":1}
I20260812 19:01:44.535619 15818 raft_consensus.cc:482] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:01:44.535811 15818 raft_consensus.cc:504] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e202ab34ffce4c42a67936b923e37188" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39655 } } peers { permanent_uuid: "a91f7679cd6a4d71949e6eace4c5c66e" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 37635 } } peers { permanent_uuid: "f6dd4bcff003483ba673140f1aedcf08" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 40085 } }
I20260812 19:01:44.536233 15818 leader_election.cc:288] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655), a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635)
W20260812 19:01:44.537035 15315 leader_election.cc:341] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e202ab34ffce4c42a67936b923e37188 (127.29.112.195:39655): Not found: Tablet not found: dfedadd810c94d62b2f1d1b91d625045
W20260812 19:01:44.537194 15317 leader_election.cc:341] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a91f7679cd6a4d71949e6eace4c5c66e (127.29.112.194:37635): Not found: Tablet not found: dfedadd810c94d62b2f1d1b91d625045
I20260812 19:01:44.537289 15317 leader_election.cc:302] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6dd4bcff003483ba673140f1aedcf08; no voters: a91f7679cd6a4d71949e6eace4c5c66e, e202ab34ffce4c42a67936b923e37188
I20260812 19:01:44.537477 15818 raft_consensus.cc:2726] T dfedadd810c94d62b2f1d1b91d625045 P f6dd4bcff003483ba673140f1aedcf08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:01:44.660382 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:44.659887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:39617)
0812 19:01:44.660176 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:44.660233 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":80,"threads_started":1}
I20260812 19:01:44.803292 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:44.802902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:43411)
0812 19:01:44.803151 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:44.803203 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":98,"threads_started":1}
I20260812 19:01:44.983251 15823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:44.982847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:58149)
0812 19:01:44.983108 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:44.983162 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":96,"threads_started":1}
I20260812 19:01:45.661624 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:45.661126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:48111)
0812 19:01:45.661481 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:45.661533 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":178,"threads_started":1}
I20260812 19:01:45.804426 15825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:45.804018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:55633)
0812 19:01:45.804265 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:45.804332 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260812 19:01:45.984468 15827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:45.984020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:51031)
0812 19:01:45.984296 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:45.984357 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
I20260812 19:01:46.662798 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:46.662358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:36103)
0812 19:01:46.662650 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:46.662705 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":114,"threads_started":1}
I20260812 19:01:46.805675 15829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:46.805223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:55359)
0812 19:01:46.805495 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:46.805576 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":98,"threads_started":1}
I20260812 19:01:46.985656 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:46.985227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:55927)
0812 19:01:46.985499 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:46.985565 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":94,"threads_started":1}
I20260812 19:01:47.664216 15832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:47.663638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:38201)
0812 19:01:47.663984 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:47.664064 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":103,"threads_started":1}
I20260812 19:01:47.806962 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:47.806564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:40103)
0812 19:01:47.806796 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:47.806850 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":77,"threads_started":1}
I20260812 19:01:47.986899 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:47.986460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:54461)
0812 19:01:47.986737 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:47.986802 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":77,"threads_started":1}
I20260812 19:01:48.665874 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:48.665360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.193:54723)
0812 19:01:48.665670 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:48.665738 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":86,"threads_started":1}
I20260812 19:01:48.808365 15839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:48.807923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.194:38369)
0812 19:01:48.808196 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:48.808255 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":88,"threads_started":1}
I20260812 19:01:48.973547 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 15261
I20260812 19:01:48.984117 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 15456
I20260812 19:01:48.988484 15840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:48.988041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:38735 (local address 127.29.112.195:48903)
0812 19:01:48.988336 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:48.988394 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":88,"threads_started":1}
I20260812 19:01:48.992383 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 15616
2026-08-12T19:01:48Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19466 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:01:49 dist-test-slave-6nmv krb5kdc[15847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:01:49 dist-test-slave-6nmv krb5kdc[15847](info): set up 2 sockets
Aug 12 19:01:49 dist-test-slave-6nmv krb5kdc[15847](info): commencing operation
krb5kdc: starting...
W20260812 19:01:50.124761 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:01:50 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:01:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:01:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:01:50.255587 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:33135
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:33135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:50.366598 15865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:50.366858 15865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:50.366897 15865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:50.370851 15865 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 19:01:50.370904 15865 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 19:01:50.370920 15865 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 19:01:50.370934 15865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:01:50.370952 15865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:50.371009 15865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:50.371042 15865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:01:50.371063 15865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:01:50.376286 15865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:33135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:33135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:50.377351 15865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:50.378418 15865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:50.387581 15873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:50.387759 15865 server_base.cc:1034] running on GCE node
W20260812 19:01:50.387692 15870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:50.387576 15871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:50.388460 15865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:50.389503 15865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:50.390664 15865 hybrid_clock.cc:648] HybridClock initialized: now 1786561310390645 us; error 30 us; skew 500 ppm
Aug 12 19:01:50 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561310, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:50.393492 15865 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:01:50.394428 15865 webserver.cc:460] Webserver started at http://127.29.112.254:41515/ using document root <none> and password file <none>
I20260812 19:01:50.395022 15865 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:50.395117 15865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:50.395354 15865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:50.397316 15865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "4da4c2299eb34bc6856b3643407143c6"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "f8d2ac048fc4fcfedf669ae25d8e51ba"
server_key_iv: "c1735c960149a8d83eb6429591008fbe"
server_key_version: "encryptionkey@0"
I20260812 19:01:50.397938 15865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "4da4c2299eb34bc6856b3643407143c6"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "f8d2ac048fc4fcfedf669ae25d8e51ba"
server_key_iv: "c1735c960149a8d83eb6429591008fbe"
server_key_version: "encryptionkey@0"
I20260812 19:01:50.401608 15865 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:01:50.404054 15881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:50.405260 15865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:01:50.405403 15865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "4da4c2299eb34bc6856b3643407143c6"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "f8d2ac048fc4fcfedf669ae25d8e51ba"
server_key_iv: "c1735c960149a8d83eb6429591008fbe"
server_key_version: "encryptionkey@0"
I20260812 19:01:50.405524 15865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:50.434492 15865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:50.437438 15865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:50.437654 15865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:50.444897 15865 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:33135
I20260812 19:01:50.445225 15940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:33135 every 8 connection(s)
I20260812 19:01:50.445925 15865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:01:50.448734 15941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:50.452806 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 15865
I20260812 19:01:50.452972 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:01:50.453255 30147 external_mini_cluster.cc:1428] Setting key d2f8862ea5eed6d4f54cb0c877a47b90
I20260812 19:01:50.454627 15941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Bootstrap starting.
I20260812 19:01:50.457051 15941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:50.457892 15941 log.cc:824] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:50.460080 15941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: No bootstrap required, opened a new log
Aug 12 19:01:50 dist-test-slave-6nmv krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:50.464677 15941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:01:50.464898 15941 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:50.464959 15941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4da4c2299eb34bc6856b3643407143c6, State: Initialized, Role: FOLLOWER
I20260812 19:01:50.465427 15941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:01:50.465585 15941 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:01:50.465673 15941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:01:50.465797 15941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:50.466727 15941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:01:50.467087 15941 leader_election.cc:302] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4da4c2299eb34bc6856b3643407143c6; no voters:
I20260812 19:01:50.467487 15941 leader_election.cc:288] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:01:50.467586 15947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:01:50.467815 15947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 1 LEADER]: Becoming Leader. State: Replica: 4da4c2299eb34bc6856b3643407143c6, State: Running, Role: LEADER
I20260812 19:01:50.468181 15947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:01:50.468537 15945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:50.454706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55096 (local address 127.29.112.254:33135)
0812 19:01:50.455138 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:50.455153 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:50.455187 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:01:50.455791 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:50.455806 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:50.456145 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:50.456408 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:50.457365 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:50.458609 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:50.459745 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:50.460264 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:50.460375 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:50.463369 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:50.463374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:50.463387 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:50.463414 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:50.465303 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:50.465799 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:50.465804 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:50.465832 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:50.466018 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:50.466320 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:50.466324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:50.466325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:50.466721 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:50.466936 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:50.467197 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 19:01:50.467535 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":120,"threads_started":1}
I20260812 19:01:50.468918 15941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:01:50.469415 15947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4da4c2299eb34bc6856b3643407143c6. Latest consensus state: current_term: 1 leader_uuid: "4da4c2299eb34bc6856b3643407143c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } } }
I20260812 19:01:50.469622 15947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:50.469897 15948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4da4c2299eb34bc6856b3643407143c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } } }
I20260812 19:01:50.469995 15948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: This master's current role is: LEADER
I20260812 19:01:50.470638 15952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:01:50.474354 15952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:01:50.479665 15952 catalog_manager.cc:1347] Generated new cluster ID: 9ae25edecdf34416807e143f5e26c100
I20260812 19:01:50.479733 15952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:01:50.511722 15952 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:01:50.512687 15952 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:01:50.527082 15952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Generated new TSK 0
I20260812 19:01:50.527837 15952 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 19:01:50.566951 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33135
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:50.703617 15973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:50.703866 15973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:50.703908 15973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:50.707813 15973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:50.707862 15973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:50.707937 15973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:01:50.713119 15973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:50.714138 15973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:50.715060 15973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:50.724478 15979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:50.724527 15984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:50.724491 15981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:50.724747 15973 server_base.cc:1034] running on GCE node
I20260812 19:01:50.725291 15973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:50.725999 15973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:50.727167 15973 hybrid_clock.cc:648] HybridClock initialized: now 1786561310727153 us; error 27 us; skew 500 ppm
Aug 12 19:01:50 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561310, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:50.729976 15973 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:01:50.731114 15973 webserver.cc:460] Webserver started at http://127.29.112.193:42683/ using document root <none> and password file <none>
I20260812 19:01:50.731842 15973 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:50.731930 15973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:50.732194 15973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:50.734632 15973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "d537b3a1edb54d458c7d53e289f329ed"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "57da24c307d2e138442cea736f693962"
server_key_iv: "2b10cd5d0c38a6b044532b8530560623"
server_key_version: "encryptionkey@0"
I20260812 19:01:50.735128 15973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "d537b3a1edb54d458c7d53e289f329ed"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "57da24c307d2e138442cea736f693962"
server_key_iv: "2b10cd5d0c38a6b044532b8530560623"
server_key_version: "encryptionkey@0"
I20260812 19:01:50.738688 15973 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:01:50.740994 15991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:50.742264 15973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:01:50.742357 15973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "d537b3a1edb54d458c7d53e289f329ed"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "57da24c307d2e138442cea736f693962"
server_key_iv: "2b10cd5d0c38a6b044532b8530560623"
server_key_version: "encryptionkey@0"
I20260812 19:01:50.742439 15973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:50.758950 15973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:50.761942 15973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:50.762099 15973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:50.762640 15973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:50.763417 15973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:50.763463 15973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:50.763511 15973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:50.763532 15973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:50.773064 15973 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:44355
I20260812 19:01:50.773130 16124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:44355 every 8 connection(s)
I20260812 19:01:50.774276 15973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:01:50.774641 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 15973
I20260812 19:01:50.774793 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 19:01:50.775245 30147 external_mini_cluster.cc:1428] Setting key 7df00ee92df8cb126e06c05945431348
Aug 12 19:01:50 dist-test-slave-6nmv krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561310, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
I20260812 19:01:50.791424 15945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:50.776363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:44617 (local address 127.29.112.254:33135)
0812 19:01:50.776512 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:50.776518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:50.777634 (+ 1116us) server_negotiation.cc:407] Connection header received
0812 19:01:50.779174 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:50.779177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:50.779220 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:50.779291 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:50.781124 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:50.781785 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:50.783841 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:50.784100 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:50.784190 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:50.787224 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:50.787231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:50.787237 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:50.787275 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:50.788830 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:50.789517 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:50.789519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:50.789521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:50.789600 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:50.790165 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:50.790172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:50.790177 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:50.790454 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:50.790557 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:50.791190 (+ 633us) server_negotiation.cc:299] Negotiation successful
0812 19:01:50.791296 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:01:50.792531 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:50.776928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:44617)
0812 19:01:50.777458 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:50.777515 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:50.778906 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:50.779488 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:50.779499 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:50.780133 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:50.780877 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:50.780914 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:50.781974 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:50.781981 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:50.783680 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:50.783697 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:50.784272 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:50.784277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:50.784508 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:50.785195 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:50.785223 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:50.786986 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:50.788952 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:50.788956 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:50.788967 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:50.789415 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:50.789687 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:50.789689 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:50.789690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:50.789990 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:50.790581 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:50.790588 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:50.790921 (+ 333us) client_negotiation.cc:769] Sending connection context
0812 19:01:50.791130 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 19:01:50.791507 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":156,"threads_started":1}
I20260812 19:01:50.793901 16125 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33135
I20260812 19:01:50.794152 16125 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:50.794699 16125 heartbeater.cc:511] Master 127.29.112.254:33135 requested a full tablet report, sending...
I20260812 19:01:50.796161 15902 ts_manager.cc:194] Registered new tserver with Master: d537b3a1edb54d458c7d53e289f329ed (127.29.112.193:44355)
I20260812 19:01:50.797268 15902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:44617
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 19:01:50.807169 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33135
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:50.917286 16131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:50.917640 16131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:50.917725 16131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:50.921715 16131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:50.921801 16131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:50.921913 16131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:01:50.927095 16131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:50.928138 16131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:50.929185 16131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:50.938683 16139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:50.938766 16138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:50.938678 16142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:50.938980 16131 server_base.cc:1034] running on GCE node
I20260812 19:01:50.939368 16131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:50.940013 16131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:50.941188 16131 hybrid_clock.cc:648] HybridClock initialized: now 1786561310941176 us; error 36 us; skew 500 ppm
Aug 12 19:01:50 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561310, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:50.943908 16131 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:01:50.944800 16131 webserver.cc:460] Webserver started at http://127.29.112.194:35419/ using document root <none> and password file <none>
I20260812 19:01:50.945374 16131 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:50.945436 16131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:50.945683 16131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:50.947768 16131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "8da42fd7f03f41a092d55d26dbf54780"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "76685f6f9526f8d513bca5f72f5b2889"
server_key_iv: "bb2df746902f742126d6b1a933de3611"
server_key_version: "encryptionkey@0"
I20260812 19:01:50.948328 16131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "8da42fd7f03f41a092d55d26dbf54780"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "76685f6f9526f8d513bca5f72f5b2889"
server_key_iv: "bb2df746902f742126d6b1a933de3611"
server_key_version: "encryptionkey@0"
I20260812 19:01:50.952104 16131 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260812 19:01:50.954493 16151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:50.955672 16131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 19:01:50.955804 16131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "8da42fd7f03f41a092d55d26dbf54780"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "76685f6f9526f8d513bca5f72f5b2889"
server_key_iv: "bb2df746902f742126d6b1a933de3611"
server_key_version: "encryptionkey@0"
I20260812 19:01:50.955912 16131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:50.969769 16131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:50.973276 16131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:50.973480 16131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:50.974177 16131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:50.975087 16131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:50.975136 16131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:50.975204 16131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:50.975246 16131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:50.984354 16131 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:42631
I20260812 19:01:50.984427 16282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:42631 every 8 connection(s)
I20260812 19:01:50.985265 16131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:01:50.994171 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 16131
I20260812 19:01:50.994313 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
Aug 12 19:01:50 dist-test-slave-6nmv krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561310, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:50.994658 30147 external_mini_cluster.cc:1428] Setting key 5c427545bf0cd2ff39968fdd057102a3
I20260812 19:01:50.999104 15945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:50.987120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:52377 (local address 127.29.112.254:33135)
0812 19:01:50.987269 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:50.987275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:50.988138 (+ 863us) server_negotiation.cc:407] Connection header received
0812 19:01:50.989341 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:50.989344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:50.989390 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:50.989465 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:50.990645 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:50.991162 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:50.992236 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:50.992443 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:50.992524 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:50.995203 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:50.995207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:50.995210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:50.995253 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:50.996694 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:50.997314 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:50.997316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:50.997318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:50.997425 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:50.997865 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:50.997870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:50.997871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:50.998081 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:50.998166 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:50.998765 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 19:01:50.998901 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:01:50.999805 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:50.987548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:52377)
0812 19:01:50.987990 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:50.988031 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:50.989158 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:50.989624 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:50.989630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:50.990049 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:50.990523 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:50.990536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:50.991266 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:50.991269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:50.992135 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:50.992144 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:50.992572 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:50.992575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:50.992720 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:50.993257 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:50.993273 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:50.995014 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:50.996806 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:50.996813 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:50.996829 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:50.997205 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:50.997513 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:50.997518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:50.997523 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:50.997711 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:50.998193 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:50.998203 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:50.998529 (+ 326us) client_negotiation.cc:769] Sending connection context
0812 19:01:50.998682 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 19:01:50.998917 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":113,"threads_started":1}
I20260812 19:01:51.000962 16283 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33135
I20260812 19:01:51.001226 16283 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:51.001857 16283 heartbeater.cc:511] Master 127.29.112.254:33135 requested a full tablet report, sending...
I20260812 19:01:51.003015 15902 ts_manager.cc:194] Registered new tserver with Master: 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631)
I20260812 19:01:51.003587 15902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:52377
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 19:01:51.024669 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33135
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:01:51.135890 16288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:01:51.136123 16288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:01:51.136165 16288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:01:51.140111 16288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:01:51.140164 16288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:01:51.140218 16288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:01:51.145366 16288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:01:51.146461 16288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:01:51.147579 16288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:01:51.157459 16294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:01:51.157459 16295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:51.157534 16288 server_base.cc:1034] running on GCE node
W20260812 19:01:51.157510 16298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:01:51.158211 16288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:01:51.158866 16288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:01:51.160030 16288 hybrid_clock.cc:648] HybridClock initialized: now 1786561311159999 us; error 49 us; skew 500 ppm
Aug 12 19:01:51 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561311, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:01:51.162994 16288 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:01:51.163870 16288 webserver.cc:460] Webserver started at http://127.29.112.195:35871/ using document root <none> and password file <none>
I20260812 19:01:51.164431 16288 fs_manager.cc:359] Metadata directory not provided
I20260812 19:01:51.164525 16288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:01:51.164755 16288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:01:51.166849 16288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "a2a96032e5c64cdea8a78d2d9972368f"
format_stamp: "Formatted at 2026-08-12 19:01:51 on dist-test-slave-6nmv"
server_key: "6a7529e6d1cfca6aeba86bec3d8f4a25"
server_key_iv: "1e8859d270e524e76b20ecb3461e85ab"
server_key_version: "encryptionkey@0"
I20260812 19:01:51.167451 16288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "a2a96032e5c64cdea8a78d2d9972368f"
format_stamp: "Formatted at 2026-08-12 19:01:51 on dist-test-slave-6nmv"
server_key: "6a7529e6d1cfca6aeba86bec3d8f4a25"
server_key_iv: "1e8859d270e524e76b20ecb3461e85ab"
server_key_version: "encryptionkey@0"
I20260812 19:01:51.171363 16288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:01:51.173878 16305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:51.175187 16288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:01:51.175344 16288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "a2a96032e5c64cdea8a78d2d9972368f"
format_stamp: "Formatted at 2026-08-12 19:01:51 on dist-test-slave-6nmv"
server_key: "6a7529e6d1cfca6aeba86bec3d8f4a25"
server_key_iv: "1e8859d270e524e76b20ecb3461e85ab"
server_key_version: "encryptionkey@0"
I20260812 19:01:51.175463 16288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:01:51.193634 16288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:01:51.196564 16288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:01:51.196774 16288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:01:51.197374 16288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:01:51.198278 16288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:01:51.198350 16288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:51.198421 16288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:01:51.198472 16288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:01:51.207383 16288 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:39187
I20260812 19:01:51.207406 16436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:39187 every 8 connection(s)
I20260812 19:01:51.208227 16288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:01:51.211447 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 16288
I20260812 19:01:51.211598 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 19:01:51.211957 30147 external_mini_cluster.cc:1428] Setting key 405f03ccfbe5e040c18241c617a5600f
Aug 12 19:01:51 dist-test-slave-6nmv krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561311, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:01:51.222527 15945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.210114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:48495 (local address 127.29.112.254:33135)
0812 19:01:51.210253 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:51.210257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:51.211629 (+ 1372us) server_negotiation.cc:407] Connection header received
0812 19:01:51.212743 (+ 1114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:51.212746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:51.212822 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:51.212901 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:51.214497 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.215016 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.216126 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.216360 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.216468 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.218997 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:51.219000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:51.219002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:51.219025 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:51.220510 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:51.221022 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:51.221031 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:51.221032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:51.221106 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:51.221443 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:51.221448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:51.221452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:51.221632 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:51.221743 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:51.222253 (+ 510us) server_negotiation.cc:299] Negotiation successful
0812 19:01:51.222369 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:01:51.223255 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.210512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:48495)
0812 19:01:51.211448 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:51.211507 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:51.212521 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:51.213149 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:51.213159 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:51.213697 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:01:51.214346 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.214361 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.215138 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.215141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.216000 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.216010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.216507 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.216511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.216687 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.217311 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:01:51.217326 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:01:51.218812 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:01:51.220639 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:51.220644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:51.220657 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:51.220907 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:51.221200 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:01:51.221204 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:01:51.221206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:01:51.221345 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:01:51.221741 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:01:51.221748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:01:51.222015 (+ 267us) client_negotiation.cc:769] Sending connection context
0812 19:01:51.222212 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 19:01:51.222520 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":547,"threads_started":1}
I20260812 19:01:51.224395 16437 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33135
I20260812 19:01:51.224647 16437 heartbeater.cc:464] Registering TS with master...
I20260812 19:01:51.225284 16437 heartbeater.cc:511] Master 127.29.112.254:33135 requested a full tablet report, sending...
I20260812 19:01:51.226351 15902 ts_manager.cc:194] Registered new tserver with Master: a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187)
I20260812 19:01:51.226893 15902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:48495
I20260812 19:01:51.237609 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:01:51.247956 15945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.239765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55112 (local address 127.29.112.254:33135)
0812 19:01:51.239942 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:51.239964 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:51.240139 (+ 175us) server_negotiation.cc:407] Connection header received
0812 19:01:51.240195 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:51.240200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:51.240260 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:51.240356 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:01:51.241312 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.242224 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.243109 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.243320 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.243405 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.244731 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:01:51.244734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:01:51.244736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:01:51.244759 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:01:51.246355 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:51.246731 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:51.246733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:51.246734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:51.246810 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:01:51.247153 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:01:51.247158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:01:51.247159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:01:51.247398 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:01:51.247504 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:51.247687 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 19:01:51.247774 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 19:01:51.250679 15902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:01:51.253913 15902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:01:51.267553 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.262481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39187 (local address 127.0.0.1:46894)
0812 19:01:51.262738 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:51.262750 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:51.262846 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:51.264099 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:51.264102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:51.264120 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:51.264214 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.264218 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.265225 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.265229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.266144 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.266154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.267167 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.267170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.267271 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.267291 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 19:01:51.267362 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 19:01:51.267412 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":3,"thread_start_us":103,"threads_started":1}
I20260812 19:01:51.267637 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.262930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46894 (local address 127.29.112.195:39187)
0812 19:01:51.263527 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:51.263541 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:51.263564 (+ 23us) server_negotiation.cc:407] Connection header received
0812 19:01:51.263638 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:51.263642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:51.263834 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:51.264108 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:51.264307 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.265098 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.266279 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.267063 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.267195 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.267317 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:51.267386 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 19:01:51.267497 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":131,"threads_started":1}
I20260812 19:01:51.267879 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.262051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.29.112.194:42631)
0812 19:01:51.262383 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:51.262395 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:51.262418 (+ 23us) server_negotiation.cc:407] Connection header received
0812 19:01:51.262566 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:51.262572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:51.262784 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:51.263000 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:51.263270 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.264389 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.266045 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.266805 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.266939 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.267072 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:51.267617 (+ 545us) server_negotiation.cc:299] Negotiation successful
0812 19:01:51.267752 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":111,"threads_started":1}
I20260812 19:01:51.267553 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.261895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42631 (local address 127.0.0.1:60710)
0812 19:01:51.262240 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:51.262292 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:51.262418 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:51.262981 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:51.262986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:51.263007 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:51.263147 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.263154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.264542 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.264547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.265916 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.265926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.266925 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.266928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.267092 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.267141 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 19:01:51.267239 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 19:01:51.267378 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":22,"thread_start_us":74,"threads_started":1}
I20260812 19:01:51.268646 16454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.262605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:44355 (local address 127.0.0.1:41168)
0812 19:01:51.263317 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:51.263332 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:51.263425 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:51.264271 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:51.264275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:51.264294 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:51.264379 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.264386 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.265747 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.265751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.267097 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.267109 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.268265 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.268269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.268355 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.268377 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:01:51.268462 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 19:01:51.268526 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":25,"thread_start_us":69,"threads_started":1}
I20260812 19:01:51.268723 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.262976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41168 (local address 127.29.112.193:44355)
0812 19:01:51.263689 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:51.263731 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:51.263751 (+ 20us) server_negotiation.cc:407] Connection header received
0812 19:01:51.263816 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:51.263821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:51.263981 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:51.264116 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:51.264508 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.265608 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.267227 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.268078 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.268211 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.268350 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:51.268489 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 19:01:51.268576 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":71,"threads_started":1}
I20260812 19:01:51.271101 16205 tablet_service.cc:1467] Processing CreateTablet for tablet 27ca35562d48456ba0d728fa8ea0b83b (DEFAULT_TABLE table=test-table [id=3171ed2040c24e54a4dc40acdcdb3372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:51.271140 16361 tablet_service.cc:1467] Processing CreateTablet for tablet 27ca35562d48456ba0d728fa8ea0b83b (DEFAULT_TABLE table=test-table [id=3171ed2040c24e54a4dc40acdcdb3372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:51.271301 16051 tablet_service.cc:1467] Processing CreateTablet for tablet 27ca35562d48456ba0d728fa8ea0b83b (DEFAULT_TABLE table=test-table [id=3171ed2040c24e54a4dc40acdcdb3372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:01:51.272115 16205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ca35562d48456ba0d728fa8ea0b83b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:51.272198 16051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ca35562d48456ba0d728fa8ea0b83b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:51.272213 16361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ca35562d48456ba0d728fa8ea0b83b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:01:51.277995 16457 tablet_bootstrap.cc:492] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780: Bootstrap starting.
I20260812 19:01:51.279392 16459 tablet_bootstrap.cc:492] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Bootstrap starting.
I20260812 19:01:51.281383 16459 tablet_bootstrap.cc:654] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:51.281877 16457 tablet_bootstrap.cc:654] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:51.282182 16458 tablet_bootstrap.cc:492] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f: Bootstrap starting.
I20260812 19:01:51.282795 16459 log.cc:824] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:51.282795 16457 log.cc:824] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:51.284664 16458 tablet_bootstrap.cc:654] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f: Neither blocks nor log segments found. Creating new log.
I20260812 19:01:51.285205 16457 tablet_bootstrap.cc:492] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780: No bootstrap required, opened a new log
I20260812 19:01:51.285205 16459 tablet_bootstrap.cc:492] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: No bootstrap required, opened a new log
I20260812 19:01:51.285400 16459 ts_tablet_manager.cc:1397] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260812 19:01:51.285400 16457 ts_tablet_manager.cc:1397] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260812 19:01:51.285401 16458 log.cc:824] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f: Log is configured to *not* fsync() on all Append() calls
I20260812 19:01:51.287010 16458 tablet_bootstrap.cc:492] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f: No bootstrap required, opened a new log
I20260812 19:01:51.287213 16458 ts_tablet_manager.cc:1397] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:01:51.289371 16457 raft_consensus.cc:348] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } }
I20260812 19:01:51.289371 16459 raft_consensus.cc:348] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } }
I20260812 19:01:51.289644 16457 raft_consensus.cc:374] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:51.289644 16459 raft_consensus.cc:374] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:51.289705 16459 raft_consensus.cc:729] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d537b3a1edb54d458c7d53e289f329ed, State: Initialized, Role: FOLLOWER
I20260812 19:01:51.289712 16457 raft_consensus.cc:729] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da42fd7f03f41a092d55d26dbf54780, State: Initialized, Role: FOLLOWER
I20260812 19:01:51.289985 16458 raft_consensus.cc:348] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } }
I20260812 19:01:51.290241 16458 raft_consensus.cc:374] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:01:51.290187 16459 consensus_queue.cc:260] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } }
I20260812 19:01:51.290187 16457 consensus_queue.cc:260] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } }
I20260812 19:01:51.290313 16458 raft_consensus.cc:729] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a96032e5c64cdea8a78d2d9972368f, State: Initialized, Role: FOLLOWER
I20260812 19:01:51.290761 16458 consensus_queue.cc:260] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } }
I20260812 19:01:51.291205 16459 ts_tablet_manager.cc:1428] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:01:51.291205 16457 ts_tablet_manager.cc:1428] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 19:01:51.291576 16283 heartbeater.cc:503] Master 127.29.112.254:33135 was elected leader, sending a full tablet report...
I20260812 19:01:51.291576 16125 heartbeater.cc:503] Master 127.29.112.254:33135 was elected leader, sending a full tablet report...
I20260812 19:01:51.292768 16458 ts_tablet_manager.cc:1428] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 19:01:51.293217 16437 heartbeater.cc:503] Master 127.29.112.254:33135 was elected leader, sending a full tablet report...
I20260812 19:01:51.326467 16464 raft_consensus.cc:482] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:01:51.326663 16464 raft_consensus.cc:504] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } }
I20260812 19:01:51.327746 16464 leader_election.cc:288] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631), a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187)
I20260812 19:01:51.333073 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.328301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:37373 (local address 127.29.112.195:39187)
0812 19:01:51.328415 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:51.328439 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:51.328455 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:01:51.328528 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:51.328532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:51.328584 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:51.328679 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:51.329114 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.329932 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.331724 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.332331 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.332457 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.332518 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:51.332871 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 19:01:51.332944 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 19:01:51.333076 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.328146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:33221 (local address 127.29.112.194:42631)
0812 19:01:51.328294 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:51.328299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:51.328457 (+ 158us) server_negotiation.cc:407] Connection header received
0812 19:01:51.328557 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:51.328560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:51.328609 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:51.328688 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:01:51.329114 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.329929 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.331724 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.332340 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.332457 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.332511 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:51.332871 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 19:01:51.332944 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 19:01:51.333127 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.327996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42631 (local address 127.29.112.193:33221)
0812 19:01:51.328363 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:51.328375 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:51.328461 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:51.328760 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:51.328763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:51.328795 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:51.328997 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.329003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.330053 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.330057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.331578 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.331589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.332465 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.332469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.332709 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.332755 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 19:01:51.332841 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 19:01:51.332938 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":347,"spinlock_wait_cycles":384,"thread_start_us":132,"threads_started":1}
I20260812 19:01:51.333561 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.327972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39187 (local address 127.29.112.193:37373)
0812 19:01:51.328283 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:51.328301 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 19:01:51.328422 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:01:51.328760 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:01:51.328773 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:01:51.328800 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:01:51.328983 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.328991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.330053 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.330057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.331578 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:01:51.331589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.332465 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.332469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:01:51.332691 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.332758 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 19:01:51.332841 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 19:01:51.332938 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":180,"spinlock_wait_cycles":17920,"thread_start_us":103,"threads_started":1}
I20260812 19:01:51.333874 16228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ca35562d48456ba0d728fa8ea0b83b" candidate_uuid: "d537b3a1edb54d458c7d53e289f329ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da42fd7f03f41a092d55d26dbf54780" is_pre_election: true
I20260812 19:01:51.334268 16228 raft_consensus.cc:2454] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d537b3a1edb54d458c7d53e289f329ed in term 0.
I20260812 19:01:51.334298 16384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ca35562d48456ba0d728fa8ea0b83b" candidate_uuid: "d537b3a1edb54d458c7d53e289f329ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a96032e5c64cdea8a78d2d9972368f" is_pre_election: true
I20260812 19:01:51.334632 16384 raft_consensus.cc:2454] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d537b3a1edb54d458c7d53e289f329ed in term 0.
I20260812 19:01:51.334825 15993 leader_election.cc:302] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da42fd7f03f41a092d55d26dbf54780, d537b3a1edb54d458c7d53e289f329ed; no voters:
I20260812 19:01:51.335160 16464 raft_consensus.cc:2781] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:01:51.335258 16464 raft_consensus.cc:482] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:01:51.335300 16464 raft_consensus.cc:3037] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:51.336395 16464 raft_consensus.cc:504] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } }
I20260812 19:01:51.336793 16464 leader_election.cc:288] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 1 election: Requested vote from peers 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631), a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187)
I20260812 19:01:51.337139 16384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ca35562d48456ba0d728fa8ea0b83b" candidate_uuid: "d537b3a1edb54d458c7d53e289f329ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a96032e5c64cdea8a78d2d9972368f"
I20260812 19:01:51.337139 16228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ca35562d48456ba0d728fa8ea0b83b" candidate_uuid: "d537b3a1edb54d458c7d53e289f329ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da42fd7f03f41a092d55d26dbf54780"
I20260812 19:01:51.337258 16384 raft_consensus.cc:3037] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:51.337258 16228 raft_consensus.cc:3037] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:01:51.338581 16228 raft_consensus.cc:2454] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d537b3a1edb54d458c7d53e289f329ed in term 1.
I20260812 19:01:51.338581 16384 raft_consensus.cc:2454] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d537b3a1edb54d458c7d53e289f329ed in term 1.
I20260812 19:01:51.338945 15996 leader_election.cc:302] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2a96032e5c64cdea8a78d2d9972368f, d537b3a1edb54d458c7d53e289f329ed; no voters:
I20260812 19:01:51.339201 16464 raft_consensus.cc:2781] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:01:51.339460 16464 raft_consensus.cc:686] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 1 LEADER]: Becoming Leader. State: Replica: d537b3a1edb54d458c7d53e289f329ed, State: Running, Role: LEADER
I20260812 19:01:51.339810 16464 consensus_queue.cc:237] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } }
I20260812 19:01:51.343255 15901 catalog_manager.cc:5495] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed reported cstate change: term changed from 0 to 1, leader changed from <none> to d537b3a1edb54d458c7d53e289f329ed (127.29.112.193). New cstate: current_term: 1 leader_uuid: "d537b3a1edb54d458c7d53e289f329ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } health_report { overall_health: UNKNOWN } } }
I20260812 19:01:51.367380 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.363998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41174 (local address 127.29.112.193:44355)
0812 19:01:51.364122 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 19:01:51.364125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:01:51.364139 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:01:51.364226 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:01:51.364228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:01:51.364284 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:01:51.364375 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:01:51.364610 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.365107 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.366001 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:01:51.366240 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:01:51.366340 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:01:51.366616 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:01:51.367023 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:01:51.367145 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 19:01:51.367198 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 19:01:51.367253 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:01:51.373618 16384 raft_consensus.cc:1261] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f [term 1 FOLLOWER]: Refusing update from remote peer d537b3a1edb54d458c7d53e289f329ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:01:51.373634 16228 raft_consensus.cc:1261] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780 [term 1 FOLLOWER]: Refusing update from remote peer d537b3a1edb54d458c7d53e289f329ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:01:51.374482 16469 consensus_queue.cc:1035] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:01:51.374698 16464 consensus_queue.cc:1035] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:01:51.382830 16475 mvcc.cc:205] Tried to move back new op lower bound from 7317755131378659328 to 7317755131250601984. Current Snapshot: MvccSnapshot[applied={T|T < 7317755131378659328}]
I20260812 19:01:51.383571 16476 mvcc.cc:205] Tried to move back new op lower bound from 7317755131378659328 to 7317755131250601984. Current Snapshot: MvccSnapshot[applied={T|T < 7317755131378659328}]
I20260812 19:01:51.391664 15900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:01:51.391809 15900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:01:51.394323 15900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Sending DeleteTablet for 3 replicas of tablet 27ca35562d48456ba0d728fa8ea0b83b
I20260812 19:01:51.395165 16205 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ca35562d48456ba0d728fa8ea0b83b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:51 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:60710
I20260812 19:01:51.395164 16051 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ca35562d48456ba0d728fa8ea0b83b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:51 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:41168
I20260812 19:01:51.395390 16361 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ca35562d48456ba0d728fa8ea0b83b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:51 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:46894
I20260812 19:01:51.395790 16486 tablet_replica.cc:330] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: stopping tablet replica
I20260812 19:01:51.395898 16488 tablet_replica.cc:330] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f: stopping tablet replica
I20260812 19:01:51.396164 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 15973
I20260812 19:01:51.396193 16488 raft_consensus.cc:2229] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:01:51.396193 16486 raft_consensus.cc:2229] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:01:51.399806 16488 raft_consensus.cc:2258] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:01:51.401381 16488 ts_tablet_manager.cc:1905] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:01:51.404767 16488 ts_tablet_manager.cc:1918] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:01:51.404848 16488 log.cc:1196] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/27ca35562d48456ba0d728fa8ea0b83b
I20260812 19:01:51.405145 16488 ts_tablet_manager.cc:1939] T 27ca35562d48456ba0d728fa8ea0b83b P a2a96032e5c64cdea8a78d2d9972368f: Deleting consensus metadata
I20260812 19:01:51.406085 16485 tablet_replica.cc:330] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780: stopping tablet replica
I20260812 19:01:51.406462 15884 catalog_manager.cc:4841] TS a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187): tablet 27ca35562d48456ba0d728fa8ea0b83b (table test-table [id=3171ed2040c24e54a4dc40acdcdb3372]) successfully deleted
I20260812 19:01:51.406481 16485 raft_consensus.cc:2229] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:01:51.406872 16485 raft_consensus.cc:2258] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:01:51.408453 16485 ts_tablet_manager.cc:1905] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:01:51.412202 16485 ts_tablet_manager.cc:1918] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:01:51.412626 16485 log.cc:1196] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/27ca35562d48456ba0d728fa8ea0b83b
W20260812 19:01:51.412797 15882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:44355 (error 108)
I20260812 19:01:51.413053 16485 ts_tablet_manager.cc:1939] T 27ca35562d48456ba0d728fa8ea0b83b P 8da42fd7f03f41a092d55d26dbf54780: Deleting consensus metadata
I20260812 19:01:51.413771 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 16131
I20260812 19:01:51.413949 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.413595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:44355 (local address 127.0.0.1:41186)
0812 19:01:51.413729 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:51.413801 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:44355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 19:01:51.420996 15882 catalog_manager.cc:4584] TS d537b3a1edb54d458c7d53e289f329ed (127.29.112.193:44355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 27ca35562d48456ba0d728fa8ea0b83b: Network error: Client connection negotiation failed: client connection to 127.29.112.193:44355: connect: Connection refused (error 111)
I20260812 19:01:51.427760 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 16288
I20260812 19:01:51.432111 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:01:51.427985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42631 (local address 127.0.0.1:60726)
0812 19:01:51.431931 (+ 3946us) negotiation.cc:107] Waiting for socket to connect
0812 19:01:51.431992 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.194:42631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3853}
I20260812 19:01:51.436535 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 15865
I20260812 19:02:11.444832 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:33135
--webserver_interface=127.29.112.254
--webserver_port=41515
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:33135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:11.556942 16490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:11.557214 16490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:11.557252 16490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:11.561146 16490 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 19:02:11.561195 16490 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 19:02:11.561210 16490 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 19:02:11.561223 16490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:02:11.561285 16490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:11.561316 16490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:11.561348 16490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:02:11.561380 16490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:02:11.566253 16490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:33135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:33135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=41515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:11.567284 16490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:11.568262 16490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:11.582231 16500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:11.582231 16496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:11.582286 16498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:11.582491 16490 server_base.cc:1034] running on GCE node
I20260812 19:02:11.583137 16490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:11.584278 16490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:11.585472 16490 hybrid_clock.cc:648] HybridClock initialized: now 1786561331585459 us; error 28 us; skew 500 ppm
Aug 12 19:02:11 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561331, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:11.593158 16490 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:02:11.594355 16490 webserver.cc:460] Webserver started at http://127.29.112.254:41515/ using document root <none> and password file <none>
I20260812 19:02:11.595011 16490 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:11.595130 16490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:11.599261 16490 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:02:11.601670 16514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:11.603119 16490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:02:11.603262 16490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "4da4c2299eb34bc6856b3643407143c6"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "f8d2ac048fc4fcfedf669ae25d8e51ba"
server_key_iv: "c1735c960149a8d83eb6429591008fbe"
server_key_version: "encryptionkey@0"
I20260812 19:02:11.603889 16490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:11.615341 16490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:11.618141 16490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:11.618351 16490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:11.625532 16490 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:33135
I20260812 19:02:11.625874 16572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:33135 every 8 connection(s)
I20260812 19:02:11.626674 16490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:02:11.631075 16575 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:02:11.632254 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 16490
I20260812 19:02:11.632750 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:44355
--local_ip_for_outbound_sockets=127.29.112.193
--tserver_master_addrs=127.29.112.254:33135
--webserver_port=42683
--webserver_interface=127.29.112.193
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:02:11.633102 16575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Bootstrap starting.
I20260812 19:02:11.650985 16575 log.cc:824] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:11.659646 16575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:02:11.660049 16575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Bootstrap complete.
I20260812 19:02:11.664151 16575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:02:11.664537 16575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4da4c2299eb34bc6856b3643407143c6, State: Initialized, Role: FOLLOWER
I20260812 19:02:11.665051 16575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:02:11.665170 16575 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:02:11.665266 16575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:02:11.665356 16575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:02:11.667270 16575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:02:11.667619 16575 leader_election.cc:302] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4da4c2299eb34bc6856b3643407143c6; no voters:
I20260812 19:02:11.667963 16575 leader_election.cc:288] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 19:02:11.668056 16580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:02:11.668254 16580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 2 LEADER]: Becoming Leader. State: Replica: 4da4c2299eb34bc6856b3643407143c6, State: Running, Role: LEADER
I20260812 19:02:11.668612 16580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:02:11.669221 16575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:02:11.669533 16580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4da4c2299eb34bc6856b3643407143c6. Latest consensus state: current_term: 2 leader_uuid: "4da4c2299eb34bc6856b3643407143c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } } }
I20260812 19:02:11.669747 16580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:11.669945 16581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4da4c2299eb34bc6856b3643407143c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } } }
I20260812 19:02:11.670143 16581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:11.670352 16585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:02:11.675259 16585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:02:11.675958 16585 catalog_manager.cc:1259] Loaded cluster ID: 9ae25edecdf34416807e143f5e26c100
I20260812 19:02:11.676034 16585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:02:11.678591 16585 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:02:11.679316 16585 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Loaded TSK: 0
I20260812 19:02:11.705518 16585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Generated new TSK 1
I20260812 19:02:11.705892 16585 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 19:02:11.750499 16577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:11.750792 16577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:11.750866 16577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:11.754899 16577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:11.754977 16577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:11.755069 16577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:02:11.760242 16577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:44355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=42683
--tserver_master_addrs=127.29.112.254:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:11.761260 16577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:11.762333 16577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:11.772964 16609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:11.772949 16610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:11.772953 16612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:11.773061 16577 server_base.cc:1034] running on GCE node
I20260812 19:02:11.773568 16577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:11.774302 16577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:11.775501 16577 hybrid_clock.cc:648] HybridClock initialized: now 1786561331775475 us; error 49 us; skew 500 ppm
Aug 12 19:02:11 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561331, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:11.778357 16577 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:02:11.779318 16577 webserver.cc:460] Webserver started at http://127.29.112.193:42683/ using document root <none> and password file <none>
I20260812 19:02:11.779904 16577 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:11.779981 16577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:11.784052 16577 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:02:11.786484 16620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:11.787884 16577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:02:11.787988 16577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "d537b3a1edb54d458c7d53e289f329ed"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "57da24c307d2e138442cea736f693962"
server_key_iv: "2b10cd5d0c38a6b044532b8530560623"
server_key_version: "encryptionkey@0"
I20260812 19:02:11.788597 16577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:11.809005 16577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:11.812492 16577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:11.812719 16577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:11.813396 16577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:11.814651 16628 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 19:02:11.819033 16577 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 19:02:11.819099 16577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 19:02:11.819171 16577 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 19:02:11.821002 16577 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 19:02:11.821062 16577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 19:02:11.821354 16628 tablet_bootstrap.cc:492] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Bootstrap starting.
I20260812 19:02:11.833472 16577 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:44355
I20260812 19:02:11.833537 16751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:44355 every 8 connection(s)
I20260812 19:02:11.834818 16577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:02:11.840534 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 16577
I20260812 19:02:11.841073 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:42631
--local_ip_for_outbound_sockets=127.29.112.194
--tserver_master_addrs=127.29.112.254:33135
--webserver_port=35419
--webserver_interface=127.29.112.194
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:02:11.844851 16628 log.cc:824] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:02:11 dist-test-slave-6nmv krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561331, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:11.853946 16628 tablet_bootstrap.cc:492] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:02:11.854362 16628 tablet_bootstrap.cc:492] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Bootstrap complete.
I20260812 19:02:11.855460 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:11.837070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:38897)
0812 19:02:11.837600 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:11.837641 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:11.838552 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:11.839712 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:11.839719 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:11.840148 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:11.840803 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:11.840827 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:11.843063 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:11.843070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:11.844398 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:11.844408 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:11.845535 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:11.845542 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:11.845757 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:11.846536 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:11.846559 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:11.848751 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:11.851700 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:11.851708 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:11.851723 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:11.852139 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:11.852582 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:11.852588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:11.852591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:11.852790 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:11.853537 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:11.853545 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:11.853898 (+ 353us) client_negotiation.cc:769] Sending connection context
0812 19:02:11.854135 (+ 237us) client_negotiation.cc:240] Negotiation successful
0812 19:02:11.854515 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":166,"threads_started":1}
I20260812 19:02:11.855681 16628 ts_tablet_manager.cc:1397] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Time spent bootstrapping tablet: real 0.035s user 0.024s sys 0.008s
I20260812 19:02:11.857244 16754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:11.837036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:38897 (local address 127.29.112.254:33135)
0812 19:02:11.837402 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:11.837423 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:11.837765 (+ 342us) server_negotiation.cc:407] Connection header received
0812 19:02:11.838961 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:11.838972 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:11.839271 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:11.839669 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:11.841031 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:11.842847 (+ 1816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:11.844563 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:11.845358 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:11.845583 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:11.848998 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:11.849005 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:11.849023 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:11.849061 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:11.851527 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:11.852282 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:11.852288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:11.852294 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:11.852463 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:11.852918 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:11.852922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:11.852924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:11.853396 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:11.853546 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:11.856159 (+ 2613us) server_negotiation.cc:299] Negotiation successful
0812 19:02:11.856443 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":128,"threads_started":1}
I20260812 19:02:11.858295 16628 raft_consensus.cc:348] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } }
I20260812 19:02:11.858747 16628 raft_consensus.cc:729] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d537b3a1edb54d458c7d53e289f329ed, State: Initialized, Role: FOLLOWER
I20260812 19:02:11.859367 16752 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33135
I20260812 19:02:11.859279 16628 consensus_queue.cc:260] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } } peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } }
I20260812 19:02:11.859623 16752 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:11.860178 16752 heartbeater.cc:511] Master 127.29.112.254:33135 requested a full tablet report, sending...
I20260812 19:02:11.860378 16628 ts_tablet_manager.cc:1428] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:02:11.862782 16533 ts_manager.cc:194] Registered new tserver with Master: d537b3a1edb54d458c7d53e289f329ed (127.29.112.193:44355)
I20260812 19:02:11.866961 16533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:38897
I20260812 19:02:11.869264 16752 heartbeater.cc:503] Master 127.29.112.254:33135 was elected leader, sending a full tablet report...
Aug 12 19:02:11 dist-test-slave-6nmv krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561331, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.193@KRBTEST.COM
I20260812 19:02:11.878376 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:11.865997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:44355 (local address 127.0.0.1:48458)
0812 19:02:11.866381 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:11.866437 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:11.866562 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:11.867299 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:11.867308 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:11.867719 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:11.867860 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:11.867867 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:11.870635 (+ 2768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:11.870642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:11.871515 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:11.871525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:11.872195 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:11.872199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:11.872324 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:11.872940 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:11.872953 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:11.874763 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:11.876629 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:11.876639 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:11.876644 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:11.876873 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:11.877179 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:11.877183 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:11.877184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:11.877272 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:11.877873 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:11.877879 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:11.877990 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 19:02:11.878105 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 19:02:11.878249 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":137,"threads_started":1}
I20260812 19:02:11.878532 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:11.865995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.29.112.193:44355)
0812 19:02:11.866777 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:11.866787 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:11.866805 (+ 18us) server_negotiation.cc:407] Connection header received
0812 19:02:11.866857 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:11.866860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:11.867018 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:11.867197 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:11.869432 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:11.870506 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:11.871695 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:11.872061 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:11.872193 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:11.874908 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:11.874924 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:11.874933 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:11.874954 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:11.876500 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:11.876990 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:11.877001 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:11.877003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:11.877080 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:11.877408 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:11.877411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:11.877413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:11.877720 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:11.877879 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:11.878209 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 19:02:11.878381 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":139,"threads_started":1}
I20260812 19:02:11.879525 16677 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ca35562d48456ba0d728fa8ea0b83b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:01:51 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:48458
I20260812 19:02:11.880352 16761 tablet_replica.cc:330] stopping tablet replica
I20260812 19:02:11.880640 16761 raft_consensus.cc:2229] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:02:11.880882 16761 raft_consensus.cc:2258] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:11.881875 16761 ts_tablet_manager.cc:1905] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:11.885262 16761 ts_tablet_manager.cc:1918] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:11.885341 16761 log.cc:1196] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal/wals/27ca35562d48456ba0d728fa8ea0b83b
I20260812 19:02:11.885607 16761 ts_tablet_manager.cc:1939] T 27ca35562d48456ba0d728fa8ea0b83b P d537b3a1edb54d458c7d53e289f329ed: Deleting consensus metadata
I20260812 19:02:11.887239 16515 catalog_manager.cc:4841] TS d537b3a1edb54d458c7d53e289f329ed (127.29.112.193:44355): tablet 27ca35562d48456ba0d728fa8ea0b83b (table test-table [id=3171ed2040c24e54a4dc40acdcdb3372]) successfully deleted
W20260812 19:02:11.962308 16756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:11.962577 16756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:11.962652 16756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:11.966447 16756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:11.966517 16756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:11.966610 16756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:02:11.971885 16756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:42631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=35419
--tserver_master_addrs=127.29.112.254:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:11.972856 16756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:11.973829 16756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:11.983765 16770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:11.983886 16756 server_base.cc:1034] running on GCE node
W20260812 19:02:11.983771 16768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:11.983771 16767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:11.984506 16756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:11.985131 16756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:11.986315 16756 hybrid_clock.cc:648] HybridClock initialized: now 1786561331986303 us; error 35 us; skew 500 ppm
Aug 12 19:02:11 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561331, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:11.989193 16756 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:02:11.990130 16756 webserver.cc:460] Webserver started at http://127.29.112.194:35419/ using document root <none> and password file <none>
I20260812 19:02:11.990690 16756 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:11.990790 16756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:11.994726 16756 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:02:11.996706 16780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:11.997959 16756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:02:11.998067 16756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "8da42fd7f03f41a092d55d26dbf54780"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "76685f6f9526f8d513bca5f72f5b2889"
server_key_iv: "bb2df746902f742126d6b1a933de3611"
server_key_version: "encryptionkey@0"
I20260812 19:02:11.998567 16756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:12.010540 16756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:12.013842 16756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:12.014007 16756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:12.014528 16756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:12.015285 16756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:12.015331 16756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:12.015369 16756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:12.015385 16756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:12.024530 16756 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:42631
I20260812 19:02:12.024585 16934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:42631 every 8 connection(s)
I20260812 19:02:12.025460 16756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:02:12.029844 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 16756
I20260812 19:02:12.030303 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:39187
--local_ip_for_outbound_sockets=127.29.112.195
--tserver_master_addrs=127.29.112.254:33135
--webserver_port=35871
--webserver_interface=127.29.112.195
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 19:02:12 dist-test-slave-6nmv krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561331, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:12.041098 16754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.027260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:55789 (local address 127.29.112.254:33135)
0812 19:02:12.027407 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:12.027411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:12.028259 (+ 848us) server_negotiation.cc:407] Connection header received
0812 19:02:12.029624 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:12.029628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:12.029695 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:12.029796 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:12.031207 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.032065 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.034004 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.034232 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.034313 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.037279 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:12.037283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:12.037285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:12.037310 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:12.038994 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:12.039575 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:12.039580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:12.039582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:12.039669 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:12.040011 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:12.040015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:12.040016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:12.040199 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:12.040307 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:12.040794 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 19:02:12.040927 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:02:12.041911 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.027656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:55789)
0812 19:02:12.028107 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:12.028152 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:12.029378 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:12.030063 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:12.030071 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:12.030541 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:12.031052 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.031067 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.032227 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.032233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.033880 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.033896 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.034396 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.034403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.034668 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.035289 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:12.035317 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:12.037102 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:12.039205 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:12.039211 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:12.039231 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:12.039470 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:12.039761 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:12.039766 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:12.039768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:12.039906 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:12.040305 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:12.040312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:12.040548 (+ 236us) client_negotiation.cc:769] Sending connection context
0812 19:02:12.040731 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 19:02:12.041050 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":106,"threads_started":1}
I20260812 19:02:12.043123 16935 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33135
I20260812 19:02:12.043406 16935 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:12.043967 16935 heartbeater.cc:511] Master 127.29.112.254:33135 requested a full tablet report, sending...
I20260812 19:02:12.045092 16533 ts_manager.cc:194] Registered new tserver with Master: 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631)
I20260812 19:02:12.045898 16533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:55789
W20260812 19:02:12.142024 16938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:12.142256 16938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:12.142295 16938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:12.146075 16938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:12.146128 16938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:12.146198 16938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:02:12.150929 16938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:39187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=35871
--tserver_master_addrs=127.29.112.254:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:12.151881 16938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:12.152935 16938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:12.162431 16943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:12.162575 16938 server_base.cc:1034] running on GCE node
W20260812 19:02:12.162509 16944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:12.162530 16946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:12.163115 16938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:12.163722 16938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:12.164901 16938 hybrid_clock.cc:648] HybridClock initialized: now 1786561332164883 us; error 29 us; skew 500 ppm
Aug 12 19:02:12 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561332, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:12.168011 16938 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:02:12.168869 16938 webserver.cc:460] Webserver started at http://127.29.112.195:35871/ using document root <none> and password file <none>
I20260812 19:02:12.169466 16938 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:12.169544 16938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:12.173609 16938 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:02:12.175696 16959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:12.176905 16938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 19:02:12.177026 16938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "a2a96032e5c64cdea8a78d2d9972368f"
format_stamp: "Formatted at 2026-08-12 19:01:51 on dist-test-slave-6nmv"
server_key: "6a7529e6d1cfca6aeba86bec3d8f4a25"
server_key_iv: "1e8859d270e524e76b20ecb3461e85ab"
server_key_version: "encryptionkey@0"
I20260812 19:02:12.177579 16938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:12.188840 16938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:12.192233 16938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:12.192430 16938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:12.193033 16938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:12.193943 16938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:12.193992 16938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:12.194054 16938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:12.194090 16938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:12.202983 16938 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:39187
I20260812 19:02:12.203018 17089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:39187 every 8 connection(s)
I20260812 19:02:12.204034 16938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:02:12.208257 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 16938
Aug 12 19:02:12 dist-test-slave-6nmv krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561332, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:12.217072 17094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.208953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55764 (local address 127.29.112.254:33135)
0812 19:02:12.209149 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:12.209152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:12.209415 (+ 263us) server_negotiation.cc:407] Connection header received
0812 19:02:12.209549 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:12.209552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:12.209595 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:12.209700 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:12.210562 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.211034 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.212030 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.212238 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.212314 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.213765 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:12.213768 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:12.213771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:12.213797 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:12.215412 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:12.215835 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:12.215845 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:12.215847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:12.215922 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:12.216252 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:12.216260 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:12.216261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:12.216478 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:12.216695 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:12.216795 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 19:02:12.216919 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"spinlock_wait_cycles":9984,"thread_start_us":80,"threads_started":1}
I20260812 19:02:12.218799 16754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.205722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:50571 (local address 127.29.112.254:33135)
0812 19:02:12.205888 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:12.205893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:12.206834 (+ 941us) server_negotiation.cc:407] Connection header received
0812 19:02:12.207911 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:12.207914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:12.207995 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:12.208092 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:12.209477 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.210201 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.211595 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.211818 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.211898 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.214851 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:12.214854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:12.214856 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:12.214888 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:12.216368 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:12.217066 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:12.217069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:12.217070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:12.217186 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:12.217630 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:12.217633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:12.217634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:12.217863 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:12.217962 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:12.218529 (+ 567us) server_negotiation.cc:299] Negotiation successful
0812 19:02:12.218647 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54,"spinlock_wait_cycles":1792}
I20260812 19:02:12.219537 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.206140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:50571)
0812 19:02:12.206556 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:12.206596 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:12.207731 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:12.208242 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:12.208248 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:12.208630 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:12.209330 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.209355 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.210353 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.210357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.211467 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.211479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.211968 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.211975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.212150 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.212754 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:12.212780 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:12.214707 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:12.216478 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:12.216491 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:12.216507 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:12.216836 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:12.217298 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:12.217302 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:12.217304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:12.217482 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:12.217978 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:12.217989 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:12.218283 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 19:02:12.218454 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 19:02:12.218690 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":102,"threads_started":1}
I20260812 19:02:12.220532 17090 heartbeater.cc:347] Connected to a master server at 127.29.112.254:33135
I20260812 19:02:12.220731 17090 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:12.221127 17090 heartbeater.cc:511] Master 127.29.112.254:33135 requested a full tablet report, sending...
I20260812 19:02:12.222088 16533 ts_manager.cc:194] Registered new tserver with Master: a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187)
I20260812 19:02:12.222656 16533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:50571
I20260812 19:02:12.230765 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:02:12.234743 16754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.231384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.29.112.254:33135)
0812 19:02:12.231527 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:12.231532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:12.231796 (+ 264us) server_negotiation.cc:407] Connection header received
0812 19:02:12.231909 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:12.231911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:12.232009 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:12.232095 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:02:12.232298 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.232797 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.233613 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.233872 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.233945 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.234240 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:02:12.234381 (+ 141us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 19:02:12.234595 (+ 214us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55766: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 19:02:12.234824 16754 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55766: token expired
W20260812 19:02:12.235149 16444 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.112.254:33135)
I20260812 19:02:12.242062 16754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.235513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.29.112.254:33135)
0812 19:02:12.235626 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:12.235629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:12.235641 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:02:12.235722 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:12.235724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:12.235759 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:12.235833 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:12.236386 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.236821 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.237524 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.237777 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.237920 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.239215 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:12.239218 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:12.239220 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:12.239240 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:12.240728 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:12.241084 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:12.241086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:12.241087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:12.241160 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:12.241379 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:12.241381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:12.241382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:12.241531 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:12.241627 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:12.241865 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 19:02:12.241952 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:02:12.244396 16533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:02:12.247722 16533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:02:12.255481 16677 tablet_service.cc:1467] Processing CreateTablet for tablet b543c48b7993494b88b9efc00a23f4ab (DEFAULT_TABLE table=test-table [id=1f043bd68b9a480c8b2c32fc5526478d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:12.256150 16677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b543c48b7993494b88b9efc00a23f4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:12.259105 16628 tablet_bootstrap.cc:492] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: Bootstrap starting.
I20260812 19:02:12.259541 17103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.254548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42631 (local address 127.0.0.1:52922)
0812 19:02:12.254819 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:12.254834 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:12.254920 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:12.255598 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:12.255601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:12.255620 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:12.255704 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.255710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.256905 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.256908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.258115 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.258127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.259129 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.259149 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.259233 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.259281 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 19:02:12.259335 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 19:02:12.259401 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
I20260812 19:02:12.260076 16628 tablet_bootstrap.cc:654] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:12.260092 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.254169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39187 (local address 127.0.0.1:48024)
0812 19:02:12.254273 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:12.254289 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:12.254362 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:12.255251 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:12.255253 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:12.255272 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:12.255372 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.255378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.256515 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.256518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.258534 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.258544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.259657 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.259661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.259761 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.259782 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:02:12.259880 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 19:02:12.259964 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":56}
I20260812 19:02:12.260198 17105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.254810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52922 (local address 127.29.112.194:42631)
0812 19:02:12.255177 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:12.255191 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:12.255213 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:02:12.255281 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:12.255286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:12.255455 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:12.255611 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:12.255796 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.256755 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.258243 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.258966 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.259759 (+ 793us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.259876 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:12.259928 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 19:02:12.260046 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":123,"threads_started":1}
I20260812 19:02:12.260542 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.254335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48024 (local address 127.29.112.195:39187)
0812 19:02:12.254740 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:12.254752 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:12.254777 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:02:12.254855 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:12.254861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:12.255094 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:12.255276 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:12.255469 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.256403 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.258671 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.259504 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.259671 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.259803 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:12.260309 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 19:02:12.260405 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":112,"threads_started":1}
I20260812 19:02:12.261662 16628 tablet_bootstrap.cc:492] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: No bootstrap required, opened a new log
I20260812 19:02:12.261754 16628 ts_tablet_manager.cc:1397] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 19:02:12.262192 16628 raft_consensus.cc:348] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:12.262326 16628 raft_consensus.cc:374] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:12.262364 16628 raft_consensus.cc:729] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d537b3a1edb54d458c7d53e289f329ed, State: Initialized, Role: FOLLOWER
I20260812 19:02:12.262566 16628 consensus_queue.cc:260] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:12.262944 16628 ts_tablet_manager.cc:1428] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 19:02:12.263461 16839 tablet_service.cc:1467] Processing CreateTablet for tablet b543c48b7993494b88b9efc00a23f4ab (DEFAULT_TABLE table=test-table [id=1f043bd68b9a480c8b2c32fc5526478d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:12.263477 17008 tablet_service.cc:1467] Processing CreateTablet for tablet b543c48b7993494b88b9efc00a23f4ab (DEFAULT_TABLE table=test-table [id=1f043bd68b9a480c8b2c32fc5526478d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:12.264567 16839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b543c48b7993494b88b9efc00a23f4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:12.264571 17008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b543c48b7993494b88b9efc00a23f4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:12.271225 17107 tablet_bootstrap.cc:492] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780: Bootstrap starting.
I20260812 19:02:12.271286 17109 tablet_bootstrap.cc:492] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f: Bootstrap starting.
I20260812 19:02:12.273782 17109 tablet_bootstrap.cc:654] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:12.273947 17107 tablet_bootstrap.cc:654] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:12.274623 17109 log.cc:824] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:12.274679 17107 log.cc:824] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:12.276352 17109 tablet_bootstrap.cc:492] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f: No bootstrap required, opened a new log
I20260812 19:02:12.276396 17107 tablet_bootstrap.cc:492] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780: No bootstrap required, opened a new log
I20260812 19:02:12.276559 17109 ts_tablet_manager.cc:1397] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:02:12.276571 17107 ts_tablet_manager.cc:1397] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 19:02:12.279629 17107 raft_consensus.cc:348] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:12.279838 17107 raft_consensus.cc:374] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:12.279635 17109 raft_consensus.cc:348] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:12.279891 17109 raft_consensus.cc:374] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:12.279906 17107 raft_consensus.cc:729] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da42fd7f03f41a092d55d26dbf54780, State: Initialized, Role: FOLLOWER
I20260812 19:02:12.279951 17109 raft_consensus.cc:729] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a96032e5c64cdea8a78d2d9972368f, State: Initialized, Role: FOLLOWER
I20260812 19:02:12.280354 17107 consensus_queue.cc:260] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:12.280352 17109 consensus_queue.cc:260] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:12.280967 16935 heartbeater.cc:503] Master 127.29.112.254:33135 was elected leader, sending a full tablet report...
I20260812 19:02:12.280992 17090 heartbeater.cc:503] Master 127.29.112.254:33135 was elected leader, sending a full tablet report...
I20260812 19:02:12.281345 17109 ts_tablet_manager.cc:1428] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260812 19:02:12.281368 17107 ts_tablet_manager.cc:1428] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
W20260812 19:02:12.336077 16753 tablet.cc:2369] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:02:12.339839 17112 raft_consensus.cc:482] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:12.340018 17112 raft_consensus.cc:504] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:12.341133 17112 leader_election.cc:288] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187), d537b3a1edb54d458c7d53e289f329ed (127.29.112.193:44355)
I20260812 19:02:12.345993 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.341540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:56865 (local address 127.29.112.193:44355)
0812 19:02:12.341688 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:12.341695 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:12.341936 (+ 241us) server_negotiation.cc:407] Connection header received
0812 19:02:12.342070 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:12.342074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:12.342147 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:12.342256 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:12.342489 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.343260 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.344710 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.345366 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.345485 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.345616 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:12.345776 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 19:02:12.345866 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 19:02:12.345992 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.341444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:55383 (local address 127.29.112.195:39187)
0812 19:02:12.341609 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:12.341614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:12.341629 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:02:12.341771 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:12.341774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:12.341877 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:12.341975 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:12.342275 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.343092 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.344710 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.345302 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.345450 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.345517 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:12.345776 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 19:02:12.345866 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 19:02:12.346007 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.341323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39187 (local address 127.29.112.194:55383)
0812 19:02:12.341508 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:12.341524 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:12.341630 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:12.342020 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:12.342024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:12.342042 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:12.342158 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.342162 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.343215 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.343217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.344571 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.344582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.345473 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.345476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.345609 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.345668 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 19:02:12.345739 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 19:02:12.345847 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":127}
I20260812 19:02:12.346014 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.341398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:44355 (local address 127.29.112.194:56865)
0812 19:02:12.341846 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:12.341857 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:12.341936 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:12.342270 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:12.342274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:12.342299 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:12.342381 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.342385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.343357 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.343360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.344571 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:12.344582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.345479 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.345489 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:12.345592 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.345663 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 19:02:12.345739 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 19:02:12.345847 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":76,"thread_start_us":150,"threads_started":1}
I20260812 19:02:12.346699 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b543c48b7993494b88b9efc00a23f4ab" candidate_uuid: "8da42fd7f03f41a092d55d26dbf54780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a96032e5c64cdea8a78d2d9972368f" is_pre_election: true
I20260812 19:02:12.346705 16700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b543c48b7993494b88b9efc00a23f4ab" candidate_uuid: "8da42fd7f03f41a092d55d26dbf54780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d537b3a1edb54d458c7d53e289f329ed" is_pre_election: true
I20260812 19:02:12.346987 16700 raft_consensus.cc:2454] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da42fd7f03f41a092d55d26dbf54780 in term 0.
I20260812 19:02:12.347008 17036 raft_consensus.cc:2454] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da42fd7f03f41a092d55d26dbf54780 in term 0.
I20260812 19:02:12.347564 16781 leader_election.cc:302] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8da42fd7f03f41a092d55d26dbf54780, a2a96032e5c64cdea8a78d2d9972368f, d537b3a1edb54d458c7d53e289f329ed; no voters:
I20260812 19:02:12.347851 17112 raft_consensus.cc:2781] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:02:12.347934 17112 raft_consensus.cc:482] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:02:12.347977 17112 raft_consensus.cc:3037] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:12.349020 17112 raft_consensus.cc:504] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:12.349315 17112 leader_election.cc:288] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [CANDIDATE]: Term 1 election: Requested vote from peers a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187), d537b3a1edb54d458c7d53e289f329ed (127.29.112.193:44355)
I20260812 19:02:12.349623 17036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b543c48b7993494b88b9efc00a23f4ab" candidate_uuid: "8da42fd7f03f41a092d55d26dbf54780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a96032e5c64cdea8a78d2d9972368f"
I20260812 19:02:12.349658 16700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b543c48b7993494b88b9efc00a23f4ab" candidate_uuid: "8da42fd7f03f41a092d55d26dbf54780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d537b3a1edb54d458c7d53e289f329ed"
I20260812 19:02:12.349751 17036 raft_consensus.cc:3037] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:12.349774 16700 raft_consensus.cc:3037] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:12.350772 16700 raft_consensus.cc:2454] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da42fd7f03f41a092d55d26dbf54780 in term 1.
I20260812 19:02:12.350857 17036 raft_consensus.cc:2454] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da42fd7f03f41a092d55d26dbf54780 in term 1.
I20260812 19:02:12.351186 16783 leader_election.cc:302] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da42fd7f03f41a092d55d26dbf54780, a2a96032e5c64cdea8a78d2d9972368f; no voters:
I20260812 19:02:12.351392 17112 raft_consensus.cc:2781] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:12.351687 17112 raft_consensus.cc:686] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 1 LEADER]: Becoming Leader. State: Replica: 8da42fd7f03f41a092d55d26dbf54780, State: Running, Role: LEADER
I20260812 19:02:12.352005 17112 consensus_queue.cc:237] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:12.354696 16533 catalog_manager.cc:5495] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194). New cstate: current_term: 1 leader_uuid: "8da42fd7f03f41a092d55d26dbf54780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } health_report { overall_health: UNKNOWN } } }
I20260812 19:02:12.383198 17105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.379792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52938 (local address 127.29.112.194:42631)
0812 19:02:12.379962 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:12.379967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:12.379981 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:02:12.380062 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:12.380065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:12.380130 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:12.380243 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:02:12.380443 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.380917 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.381784 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:12.382058 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:12.382154 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:12.382444 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:02:12.382834 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:02:12.382922 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:12.382979 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 19:02:12.383026 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 19:02:12.388880 16700 raft_consensus.cc:1261] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Refusing update from remote peer 8da42fd7f03f41a092d55d26dbf54780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:12.388885 17036 raft_consensus.cc:1261] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f [term 1 FOLLOWER]: Refusing update from remote peer 8da42fd7f03f41a092d55d26dbf54780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:12.389617 17115 consensus_queue.cc:1035] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:12.389837 17112 consensus_queue.cc:1035] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:12.395614 17119 mvcc.cc:205] Tried to move back new op lower bound from 7317755217457889280 to 7317755217316216832. Current Snapshot: MvccSnapshot[applied={T|T < 7317755217457889280}]
I20260812 19:02:12.398656 17120 mvcc.cc:205] Tried to move back new op lower bound from 7317755217457889280 to 7317755217316216832. Current Snapshot: MvccSnapshot[applied={T|T < 7317755217457889280}]
I20260812 19:02:12.405269 16533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:12.405402 16533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:12.407667 16533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Sending DeleteTablet for 3 replicas of tablet b543c48b7993494b88b9efc00a23f4ab
I20260812 19:02:12.408363 17008 tablet_service.cc:1514] Processing DeleteTablet for tablet b543c48b7993494b88b9efc00a23f4ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:12 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:48024
I20260812 19:02:12.408406 16839 tablet_service.cc:1514] Processing DeleteTablet for tablet b543c48b7993494b88b9efc00a23f4ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:12 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:52922
I20260812 19:02:12.408795 16677 tablet_service.cc:1514] Processing DeleteTablet for tablet b543c48b7993494b88b9efc00a23f4ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:12 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:48458
I20260812 19:02:12.408852 17133 tablet_replica.cc:330] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f: stopping tablet replica
I20260812 19:02:12.409142 17134 tablet_replica.cc:330] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: stopping tablet replica
I20260812 19:02:12.409142 17133 raft_consensus.cc:2229] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:02:12.409339 17134 raft_consensus.cc:2229] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:02:12.409409 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 16577
I20260812 19:02:12.409430 17133 raft_consensus.cc:2258] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:12.409757 17132 tablet_replica.cc:330] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780: stopping tablet replica
I20260812 19:02:12.410043 17132 raft_consensus.cc:2229] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:02:12.410328 17132 raft_consensus.cc:2258] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:12.411185 17133 ts_tablet_manager.cc:1905] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:12.412436 17132 ts_tablet_manager.cc:1905] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:12.415109 17132 ts_tablet_manager.cc:1918] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:12.415203 17132 log.cc:1196] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/b543c48b7993494b88b9efc00a23f4ab
I20260812 19:02:12.415514 17133 ts_tablet_manager.cc:1918] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:12.415581 17132 ts_tablet_manager.cc:1939] T b543c48b7993494b88b9efc00a23f4ab P 8da42fd7f03f41a092d55d26dbf54780: Deleting consensus metadata
I20260812 19:02:12.415782 17133 log.cc:1196] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/b543c48b7993494b88b9efc00a23f4ab
I20260812 19:02:12.416328 17133 ts_tablet_manager.cc:1939] T b543c48b7993494b88b9efc00a23f4ab P a2a96032e5c64cdea8a78d2d9972368f: Deleting consensus metadata
I20260812 19:02:12.416749 16515 catalog_manager.cc:4841] TS 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631): tablet b543c48b7993494b88b9efc00a23f4ab (table test-table [id=1f043bd68b9a480c8b2c32fc5526478d]) successfully deleted
I20260812 19:02:12.417941 16517 catalog_manager.cc:4841] TS a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187): tablet b543c48b7993494b88b9efc00a23f4ab (table test-table [id=1f043bd68b9a480c8b2c32fc5526478d]) successfully deleted
W20260812 19:02:12.429967 16515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:44355 (error 108)
I20260812 19:02:12.430578 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 16756
I20260812 19:02:12.432904 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:12.430816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:44355 (local address 127.0.0.1:48474)
0812 19:02:12.432713 (+ 1897us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:12.432796 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:44355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1790}
W20260812 19:02:12.433436 16515 catalog_manager.cc:4584] TS d537b3a1edb54d458c7d53e289f329ed (127.29.112.193:44355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b543c48b7993494b88b9efc00a23f4ab: Network error: Client connection negotiation failed: client connection to 127.29.112.193:44355: connect: Connection refused (error 111)
I20260812 19:02:12.439230 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 16938
I20260812 19:02:12.447824 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 16490
I20260812 19:02:32.456598 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:33135
--webserver_interface=127.29.112.254
--webserver_port=41515
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:33135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:32.572983 17136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:32.573254 17136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:32.573295 17136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:32.577412 17136 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 19:02:32.577486 17136 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 19:02:32.577502 17136 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 19:02:32.577515 17136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:02:32.577533 17136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:32.577580 17136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:32.577620 17136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:02:32.577639 17136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:02:32.583057 17136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:33135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:33135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=41515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:32.584239 17136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:32.585325 17136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:32.594395 17144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:32.594468 17141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:32.594532 17142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:32.594697 17136 server_base.cc:1034] running on GCE node
I20260812 19:02:32.595235 17136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:32.596333 17136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:32.597536 17136 hybrid_clock.cc:648] HybridClock initialized: now 1786561352597504 us; error 46 us; skew 500 ppm
Aug 12 19:02:32 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561352, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:32.606129 17136 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:02:32.607522 17136 webserver.cc:460] Webserver started at http://127.29.112.254:41515/ using document root <none> and password file <none>
I20260812 19:02:32.608369 17136 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:32.608480 17136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:32.613868 17136 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 19:02:32.617240 17151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:32.618722 17136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260812 19:02:32.618857 17136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "4da4c2299eb34bc6856b3643407143c6"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "f8d2ac048fc4fcfedf669ae25d8e51ba"
server_key_iv: "c1735c960149a8d83eb6429591008fbe"
server_key_version: "encryptionkey@0"
I20260812 19:02:32.619515 17136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:32.637403 17136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:32.640583 17136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:32.640743 17136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:32.648298 17136 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:33135
I20260812 19:02:32.648607 17213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:33135 every 8 connection(s)
I20260812 19:02:32.649453 17136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:02:32.653827 17214 sys_catalog.cc:263] Verifying existing consensus state
I20260812 19:02:32.654910 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 17136
I20260812 19:02:32.655434 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:44355
--local_ip_for_outbound_sockets=127.29.112.193
--tserver_master_addrs=127.29.112.254:33135
--webserver_port=42683
--webserver_interface=127.29.112.193
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:02:32.655942 17214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Bootstrap starting.
I20260812 19:02:32.674894 17214 log.cc:824] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:32.686411 17214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:02:32.686828 17214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Bootstrap complete.
I20260812 19:02:32.691175 17214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:02:32.691596 17214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4da4c2299eb34bc6856b3643407143c6, State: Initialized, Role: FOLLOWER
I20260812 19:02:32.692114 17214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:02:32.692265 17214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:02:32.692356 17214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:02:32.692483 17214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:02:32.694469 17214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:02:32.694808 17214 leader_election.cc:302] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4da4c2299eb34bc6856b3643407143c6; no voters:
I20260812 19:02:32.695188 17214 leader_election.cc:288] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 19:02:32.695279 17218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:02:32.695494 17218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [term 3 LEADER]: Becoming Leader. State: Replica: 4da4c2299eb34bc6856b3643407143c6, State: Running, Role: LEADER
I20260812 19:02:32.695802 17218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } }
I20260812 19:02:32.696470 17214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:02:32.696794 17218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4da4c2299eb34bc6856b3643407143c6. Latest consensus state: current_term: 3 leader_uuid: "4da4c2299eb34bc6856b3643407143c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } } }
I20260812 19:02:32.696918 17218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:32.697281 17222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:02:32.697306 17219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4da4c2299eb34bc6856b3643407143c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4da4c2299eb34bc6856b3643407143c6" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 33135 } } }
I20260812 19:02:32.697525 17219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:32.702648 17222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:02:32.703559 17222 catalog_manager.cc:1259] Loaded cluster ID: 9ae25edecdf34416807e143f5e26c100
I20260812 19:02:32.703632 17222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:02:32.706223 17222 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:02:32.706977 17222 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Loaded TSK: 0
I20260812 19:02:32.707064 17222 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4da4c2299eb34bc6856b3643407143c6: Loaded TSK: 1
F20260812 19:02:32.707849 17222 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9f594579ad google::LogMessage::Fail() at ??:0
@ 0x7f9f5945a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f9f594574de google::LogMessage::Flush() at ??:0
@ 0x7f9f59458a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9f5c11b18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9f5c113f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f9f63dbc7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f9f63d9699c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f9f63d97178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f9f63dd8df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f9f63df782b std::function<>::operator()() at ??:0
@ 0x7f9f63d96b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9f63d97f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9f63d94579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9f63dd8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9f5b6a93d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9f5b6a9c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f9f5b6ab3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9f5b69a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9f5ccde6db start_thread at ??:0
@ 0x7f9f57f5561f clone at ??:0
W20260812 19:02:32.795539 17216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:32.795912 17216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:32.795981 17216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:32.802560 17216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:32.802634 17216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:32.802735 17216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:02:32.811091 17216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:44355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=42683
--tserver_master_addrs=127.29.112.254:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:32.812541 17216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:32.813916 17216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:32.826706 17271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:32.826860 17268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:32.826706 17269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:32.827093 17216 server_base.cc:1034] running on GCE node
I20260812 19:02:32.827713 17216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:32.828490 17216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:32.829689 17216 hybrid_clock.cc:648] HybridClock initialized: now 1786561352829672 us; error 36 us; skew 500 ppm
Aug 12 19:02:32 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561352, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:32.832929 17216 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:02:32.834105 17216 webserver.cc:460] Webserver started at http://127.29.112.193:42683/ using document root <none> and password file <none>
I20260812 19:02:32.834724 17216 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:32.834828 17216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:32.839249 17216 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 19:02:32.841732 17279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:32.843189 17216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:02:32.843330 17216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "d537b3a1edb54d458c7d53e289f329ed"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "57da24c307d2e138442cea736f693962"
server_key_iv: "2b10cd5d0c38a6b044532b8530560623"
server_key_version: "encryptionkey@0"
I20260812 19:02:32.843915 17216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:32.862576 17216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:32.866173 17216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:32.866393 17216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:32.867036 17216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:32.868232 17286 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 19:02:32.873531 17216 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 19:02:32.873615 17216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260812 19:02:32.873739 17216 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 19:02:32.875559 17216 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 19:02:32.875638 17216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 19:02:32.875941 17286 tablet_bootstrap.cc:492] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: Bootstrap starting.
I20260812 19:02:32.888468 17216 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:44355
I20260812 19:02:32.888705 17419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:44355 every 8 connection(s)
I20260812 19:02:32.889516 17216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:02:32.893901 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 17216
I20260812 19:02:32.894428 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:42631
--local_ip_for_outbound_sockets=127.29.112.194
--tserver_master_addrs=127.29.112.254:33135
--webserver_port=35419
--webserver_interface=127.29.112.194
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:02:32.900288 17286 log.cc:824] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:32.907222 17286 tablet_bootstrap.cc:492] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:02:32.907656 17286 tablet_bootstrap.cc:492] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: Bootstrap complete.
I20260812 19:02:32.908874 17286 ts_tablet_manager.cc:1397] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: Time spent bootstrapping tablet: real 0.033s user 0.015s sys 0.012s
I20260812 19:02:32.911453 17286 raft_consensus.cc:348] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:32.911900 17286 raft_consensus.cc:729] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d537b3a1edb54d458c7d53e289f329ed, State: Initialized, Role: FOLLOWER
I20260812 19:02:32.912390 17286 consensus_queue.cc:260] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:32.913342 17286 ts_tablet_manager.cc:1428] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
W20260812 19:02:32.975870 17424 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:02:32.891511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:40339)
0812 19:02:32.891994 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:32.892048 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:32.892890 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:32.975062 (+ 82172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":139,"threads_started":1}
W20260812 19:02:32.976552 17421 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:33135 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.254:33135: Client connection negotiation failed: client connection to 127.29.112.254:33135: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 19:02:32.977340 17424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:32.977049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:48277)
0812 19:02:32.977197 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:32.977275 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 19:02:32.978113 17424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:32.977900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:48573)
0812 19:02:32.978012 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:32.978063 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 19:02:32.978309 17421 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:02:33.031306 17425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:33.031601 17425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:33.031742 17425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:33.035811 17425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:33.035899 17425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:33.035998 17425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:02:33.041334 17425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:42631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=35419
--tserver_master_addrs=127.29.112.254:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:33.042520 17425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:33.043553 17425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:33.053242 17434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:33.053267 17437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:33.053282 17432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:33.053501 17425 server_base.cc:1034] running on GCE node
I20260812 19:02:33.054082 17425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:33.054795 17425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:33.055964 17425 hybrid_clock.cc:648] HybridClock initialized: now 1786561353055948 us; error 28 us; skew 500 ppm
Aug 12 19:02:33 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561353, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:33.058908 17425 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:02:33.059775 17425 webserver.cc:460] Webserver started at http://127.29.112.194:35419/ using document root <none> and password file <none>
I20260812 19:02:33.060467 17425 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:33.060568 17425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:33.064841 17425 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:02:33.066970 17450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:33.068192 17425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 19:02:33.068325 17425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "8da42fd7f03f41a092d55d26dbf54780"
format_stamp: "Formatted at 2026-08-12 19:01:50 on dist-test-slave-6nmv"
server_key: "76685f6f9526f8d513bca5f72f5b2889"
server_key_iv: "bb2df746902f742126d6b1a933de3611"
server_key_version: "encryptionkey@0"
I20260812 19:02:33.068877 17425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:33.083165 17425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:33.086135 17425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:33.086344 17425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:33.086946 17425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:33.087798 17425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:33.087872 17425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:33.087947 17425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:33.087990 17425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:33.097652 17425 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:42631
I20260812 19:02:33.097675 17591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:42631 every 8 connection(s)
I20260812 19:02:33.098735 17425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:02:33.102669 17594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:33.100941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:39691)
0812 19:02:33.101426 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:33.101953 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":142,"threads_started":1}
W20260812 19:02:33.103123 17592 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:33135 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.254:33135: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
I20260812 19:02:33.103618 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 17425
I20260812 19:02:33.103766 17594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:33.103496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:34415)
0812 19:02:33.103632 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:33.103695 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 19:02:33.104154 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:39187
--local_ip_for_outbound_sockets=127.29.112.195
--tserver_master_addrs=127.29.112.254:33135
--webserver_port=35871
--webserver_interface=127.29.112.195
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:02:33.104488 17594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:33.104194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:39273)
0812 19:02:33.104353 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:33.104413 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 19:02:33.104671 17592 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 19:02:33.216158 17595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:33.216454 17595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:33.216531 17595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:33.220506 17595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:33.220588 17595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:33.220682 17595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:02:33.225847 17595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:39187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=35871
--tserver_master_addrs=127.29.112.254:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:33.226891 17595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:33.227916 17595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:33.237536 17603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:33.237512 17600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:33.237512 17601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:33.237869 17595 server_base.cc:1034] running on GCE node
I20260812 19:02:33.238394 17595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:33.239035 17595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:33.240201 17595 hybrid_clock.cc:648] HybridClock initialized: now 1786561353240166 us; error 49 us; skew 500 ppm
Aug 12 19:02:33 dist-test-slave-6nmv krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561353, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:33.243012 17595 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:02:33.243928 17595 webserver.cc:460] Webserver started at http://127.29.112.195:35871/ using document root <none> and password file <none>
I20260812 19:02:33.244516 17595 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:33.244581 17595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:33.248528 17595 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:02:33.250759 17610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:33.251998 17595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:02:33.252131 17595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "a2a96032e5c64cdea8a78d2d9972368f"
format_stamp: "Formatted at 2026-08-12 19:01:51 on dist-test-slave-6nmv"
server_key: "6a7529e6d1cfca6aeba86bec3d8f4a25"
server_key_iv: "1e8859d270e524e76b20ecb3461e85ab"
server_key_version: "encryptionkey@0"
I20260812 19:02:33.252663 17595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:33.263069 17595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:33.265923 17595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:33.266126 17595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:33.266700 17595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:33.267516 17595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:33.267563 17595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:33.267632 17595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:33.267669 17595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:33.277324 17595 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:39187
I20260812 19:02:33.277354 17748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:39187 every 8 connection(s)
I20260812 19:02:33.278409 17595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:02:33.281260 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 17595
I20260812 19:02:33.282445 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:33.280487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:53663)
0812 19:02:33.280949 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:33.281454 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":109,"threads_started":1}
W20260812 19:02:33.282924 17749 heartbeater.cc:644] Failed to heartbeat to 127.29.112.254:33135 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.254:33135: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
I20260812 19:02:33.283471 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:33.283267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:52549)
0812 19:02:33.283366 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:33.283415 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 19:02:33.284076 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:33.283890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:40607)
0812 19:02:33.283984 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:33.284028 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 19:02:33.284276 17749 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 19:02:33.979453 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:33.978965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:41201)
0812 19:02:33.979254 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:33.979313 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":83,"threads_started":1}
I20260812 19:02:34.105638 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:34.105238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:50077)
0812 19:02:34.105485 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:34.105539 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":74,"threads_started":1}
I20260812 19:02:34.285260 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:34.284850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:58499)
0812 19:02:34.285110 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:34.285160 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
I20260812 19:02:34.342531 17758 raft_consensus.cc:482] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:34.342751 17758 raft_consensus.cc:504] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:34.343930 17758 leader_election.cc:288] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187), 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631)
Aug 12 19:02:34 dist-test-slave-6nmv krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561352, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.195@KRBTEST.COM
Aug 12 19:02:34 dist-test-slave-6nmv krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561352, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.194@KRBTEST.COM
I20260812 19:02:34.358842 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:34.344373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:34921 (local address 127.29.112.195:39187)
0812 19:02:34.344777 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:34.344809 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:34.344876 (+ 67us) server_negotiation.cc:407] Connection header received
0812 19:02:34.345854 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:34.345872 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:34.346330 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:34.346680 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:34.347872 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:34.348965 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:34.350244 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:34.350936 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:34.351118 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:34.353585 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:34.353592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:34.353608 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:34.353643 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:34.355620 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:34.356374 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:34.356378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:34.356382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:34.356517 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:34.357169 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:34.357172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:34.357174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:34.357690 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:34.357853 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:34.358372 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 19:02:34.358640 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":99,"threads_started":1}
I20260812 19:02:34.358824 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:34.344148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:39187 (local address 127.29.112.193:34921)
0812 19:02:34.344368 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:34.344386 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:34.344509 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:34.346736 (+ 2227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:34.346741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:34.347122 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:34.347669 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:34.347680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:34.349141 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:34.349144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:34.350121 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:34.350130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:34.351059 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:34.351062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:34.351219 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:34.351813 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:34.351829 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:34.353355 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:34.355744 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:34.355750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:34.355763 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:34.356236 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:34.356620 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:34.356623 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:34.356625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:34.356762 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:34.357840 (+ 1078us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:34.357851 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:34.358072 (+ 221us) client_negotiation.cc:769] Sending connection context
0812 19:02:34.358263 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 19:02:34.358640 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":39,"spinlock_wait_cycles":23680}
I20260812 19:02:34.358901 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:34.344388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:39967 (local address 127.29.112.194:42631)
0812 19:02:34.344841 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:34.344864 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:34.344898 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:02:34.345855 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:34.345865 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:34.346330 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:34.346680 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:34.347823 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:34.348965 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:34.350291 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:34.351000 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:34.351190 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:34.354024 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:34.354029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:34.354038 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:34.354068 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:34.355772 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:34.356647 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:34.356651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:34.356655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:34.356761 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:34.357146 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:34.357149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:34.357151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:34.357690 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:34.357853 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:34.358376 (+ 523us) server_negotiation.cc:299] Negotiation successful
0812 19:02:34.358706 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":127,"threads_started":1}
I20260812 19:02:34.359539 17759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:34.344203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:42631 (local address 127.29.112.193:39967)
0812 19:02:34.344556 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:34.344585 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:34.344695 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:34.347193 (+ 2498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:34.347195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:34.347439 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:34.347648 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:34.347665 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:34.349138 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:34.349141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:34.350129 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:34.350141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:34.351141 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:34.351146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:34.351239 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:34.351997 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:34.352021 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:34.353793 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:34.355888 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:34.355891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:34.355892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:34.356230 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:34.356894 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:34.356897 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:34.356899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:34.357015 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:34.357916 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:34.357919 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:34.358067 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 19:02:34.358264 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 19:02:34.358763 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":306,"spinlock_wait_cycles":19968,"thread_start_us":76,"threads_started":1}
W20260812 19:02:34.361696 17282 leader_election.cc:341] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187): Not found: Tablet not found: b543c48b7993494b88b9efc00a23f4ab
W20260812 19:02:34.361891 17280 leader_election.cc:341] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631): Not found: Tablet not found: b543c48b7993494b88b9efc00a23f4ab
I20260812 19:02:34.361943 17282 leader_election.cc:302] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d537b3a1edb54d458c7d53e289f329ed; no voters: 8da42fd7f03f41a092d55d26dbf54780, a2a96032e5c64cdea8a78d2d9972368f
I20260812 19:02:34.362232 17758 raft_consensus.cc:2726] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:02:34.980767 17762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:34.980295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:55969)
0812 19:02:34.980606 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:34.980667 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":115,"threads_started":1}
I20260812 19:02:35.106791 17763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:35.106357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:38369)
0812 19:02:35.106636 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:35.106702 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":80,"threads_started":1}
I20260812 19:02:35.286454 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:35.286055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:48869)
0812 19:02:35.286303 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:35.286362 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
I20260812 19:02:35.982093 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:35.981629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:34485)
0812 19:02:35.981932 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:35.981991 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":79,"threads_started":1}
I20260812 19:02:36.108078 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:36.107610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:49127)
0812 19:02:36.107890 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:36.107946 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":98,"threads_started":1}
I20260812 19:02:36.222389 17767 raft_consensus.cc:482] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:36.222532 17767 raft_consensus.cc:504] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:36.222934 17767 leader_election.cc:288] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187), 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631)
W20260812 19:02:36.223886 17280 leader_election.cc:341] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631): Not found: Tablet not found: b543c48b7993494b88b9efc00a23f4ab
W20260812 19:02:36.223996 17282 leader_election.cc:341] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187): Not found: Tablet not found: b543c48b7993494b88b9efc00a23f4ab
I20260812 19:02:36.224072 17282 leader_election.cc:302] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d537b3a1edb54d458c7d53e289f329ed; no voters: 8da42fd7f03f41a092d55d26dbf54780, a2a96032e5c64cdea8a78d2d9972368f
I20260812 19:02:36.224210 17767 raft_consensus.cc:2726] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:02:36.287650 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:36.287204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:41195)
0812 19:02:36.287480 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:36.287541 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":80,"threads_started":1}
I20260812 19:02:36.983284 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:36.982870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:46771)
0812 19:02:36.983130 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:36.983186 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":90,"threads_started":1}
I20260812 19:02:37.109267 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:37.108842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:52697)
0812 19:02:37.109121 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:37.109178 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":99,"threads_started":1}
I20260812 19:02:37.288769 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:37.288373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:59303)
0812 19:02:37.288624 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:37.288678 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":76,"threads_started":1}
I20260812 19:02:37.984817 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:37.984349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:38007)
0812 19:02:37.984666 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:37.984722 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":106,"threads_started":1}
I20260812 19:02:38.110893 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:38.110352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:56753)
0812 19:02:38.110724 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:38.110791 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":116,"threads_started":1}
I20260812 19:02:38.290100 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:38.289666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:40133)
0812 19:02:38.289957 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:38.290010 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":89,"threads_started":1}
I20260812 19:02:38.370585 17777 raft_consensus.cc:482] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:38.370728 17777 raft_consensus.cc:504] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:38.371246 17777 leader_election.cc:288] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187), 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631)
W20260812 19:02:38.372094 17282 leader_election.cc:341] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187): Not found: Tablet not found: b543c48b7993494b88b9efc00a23f4ab
W20260812 19:02:38.372246 17280 leader_election.cc:341] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631): Not found: Tablet not found: b543c48b7993494b88b9efc00a23f4ab
I20260812 19:02:38.372350 17280 leader_election.cc:302] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d537b3a1edb54d458c7d53e289f329ed; no voters: 8da42fd7f03f41a092d55d26dbf54780, a2a96032e5c64cdea8a78d2d9972368f
I20260812 19:02:38.372521 17777 raft_consensus.cc:2726] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:02:38.986119 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:38.985620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:57455)
0812 19:02:38.985963 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:38.986020 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":95,"threads_started":1}
I20260812 19:02:39.112257 17780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:39.111804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:42163)
0812 19:02:39.112090 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:39.112156 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":93,"threads_started":1}
I20260812 19:02:39.291231 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:39.290810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:47261)
0812 19:02:39.291085 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:39.291141 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":100,"threads_started":1}
I20260812 19:02:39.987409 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:39.986887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:40327)
0812 19:02:39.987224 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:39.987288 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":102,"threads_started":1}
I20260812 19:02:40.113655 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:40.113152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:55267)
0812 19:02:40.113498 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:40.113561 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":98,"threads_started":1}
I20260812 19:02:40.292564 17784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:40.292067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:52337)
0812 19:02:40.292401 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:40.292455 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":109,"threads_started":1}
I20260812 19:02:40.988860 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:40.988363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:40031)
0812 19:02:40.988699 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:40.988762 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":117,"threads_started":1}
I20260812 19:02:41.114804 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:41.114376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:39189)
0812 19:02:41.114654 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:41.114712 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":91,"threads_started":1}
I20260812 19:02:41.293862 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:41.293340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:43315)
0812 19:02:41.293644 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:41.293708 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":94,"threads_started":1}
I20260812 19:02:41.990263 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:41.989715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:49259)
0812 19:02:41.990078 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:41.990160 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":190,"threads_started":1}
I20260812 19:02:42.116108 17791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:42.115647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:45519)
0812 19:02:42.115916 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:42.115995 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":107,"threads_started":1}
I20260812 19:02:42.295048 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:42.294609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:40311)
0812 19:02:42.294884 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:42.294946 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260812 19:02:42.324255 17794 raft_consensus.cc:482] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:42.324404 17794 raft_consensus.cc:504] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:42.324836 17794 leader_election.cc:288] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187), 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631)
W20260812 19:02:42.325614 17282 leader_election.cc:341] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187): Not found: Tablet not found: b543c48b7993494b88b9efc00a23f4ab
W20260812 19:02:42.325722 17280 leader_election.cc:341] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631): Not found: Tablet not found: b543c48b7993494b88b9efc00a23f4ab
I20260812 19:02:42.325765 17282 leader_election.cc:302] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d537b3a1edb54d458c7d53e289f329ed; no voters: 8da42fd7f03f41a092d55d26dbf54780, a2a96032e5c64cdea8a78d2d9972368f
I20260812 19:02:42.325935 17794 raft_consensus.cc:2726] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:02:42.991605 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:42.991191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:52771)
0812 19:02:42.991452 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:42.991508 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260812 19:02:43.117380 17796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:43.116954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:60569)
0812 19:02:43.117229 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:43.117287 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":91,"threads_started":1}
I20260812 19:02:43.296296 17798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:43.295881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:53765)
0812 19:02:43.296151 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:43.296207 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":88,"threads_started":1}
I20260812 19:02:43.992995 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:43.992515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:59361)
0812 19:02:43.992801 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:43.992871 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":79,"threads_started":1}
I20260812 19:02:44.118577 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:44.118204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:60185)
0812 19:02:44.118428 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:44.118479 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":70,"threads_started":1}
I20260812 19:02:44.297448 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:44.297035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:53007)
0812 19:02:44.297297 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:44.297351 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
I20260812 19:02:44.994211 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:44.993766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:60637)
0812 19:02:44.994050 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:44.994112 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":95,"threads_started":1}
I20260812 19:02:45.119689 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:45.119294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:52089)
0812 19:02:45.119542 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:45.119599 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":83,"threads_started":1}
I20260812 19:02:45.298627 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:45.298228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:56619)
0812 19:02:45.298485 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:45.298540 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":66,"threads_started":1}
I20260812 19:02:45.995388 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:45.994982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:42055)
0812 19:02:45.995239 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:45.995293 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":98,"threads_started":1}
I20260812 19:02:46.120820 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:46.120399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:39235)
0812 19:02:46.120671 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:46.120728 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":89,"threads_started":1}
I20260812 19:02:46.300004 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:46.299572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:47885)
0812 19:02:46.299838 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:46.299899 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":88,"threads_started":1}
I20260812 19:02:46.996658 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:46.996204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:51453)
0812 19:02:46.996491 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:46.996553 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":79,"threads_started":1}
I20260812 19:02:47.122063 17811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:47.121547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:52241)
0812 19:02:47.121888 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:47.121953 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":78,"threads_started":1}
I20260812 19:02:47.301226 17812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:47.300831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:37795)
0812 19:02:47.301086 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:47.301138 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260812 19:02:47.617046 17813 raft_consensus.cc:482] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:47.617208 17813 raft_consensus.cc:504] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a96032e5c64cdea8a78d2d9972368f" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 39187 } } peers { permanent_uuid: "8da42fd7f03f41a092d55d26dbf54780" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 42631 } } peers { permanent_uuid: "d537b3a1edb54d458c7d53e289f329ed" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 44355 } }
I20260812 19:02:47.617662 17813 leader_election.cc:288] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187), 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631)
W20260812 19:02:47.618585 17282 leader_election.cc:341] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a2a96032e5c64cdea8a78d2d9972368f (127.29.112.195:39187): Not found: Tablet not found: b543c48b7993494b88b9efc00a23f4ab
W20260812 19:02:47.618711 17280 leader_election.cc:341] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8da42fd7f03f41a092d55d26dbf54780 (127.29.112.194:42631): Not found: Tablet not found: b543c48b7993494b88b9efc00a23f4ab
I20260812 19:02:47.618780 17280 leader_election.cc:302] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d537b3a1edb54d458c7d53e289f329ed; no voters: 8da42fd7f03f41a092d55d26dbf54780, a2a96032e5c64cdea8a78d2d9972368f
I20260812 19:02:47.618947 17813 raft_consensus.cc:2726] T b543c48b7993494b88b9efc00a23f4ab P d537b3a1edb54d458c7d53e289f329ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:02:47.998030 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:47.997607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.193:38723)
0812 19:02:47.997867 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:47.997935 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":77,"threads_started":1}
I20260812 19:02:48.123270 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:48.122896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.194:37647)
0812 19:02:48.123126 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:48.123180 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":62,"threads_started":1}
I20260812 19:02:48.286890 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 17216
I20260812 19:02:48.302634 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:48.302196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:33135 (local address 127.29.112.195:38993)
0812 19:02:48.302437 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:48.302501 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.254:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
I20260812 19:02:48.303112 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 17425
I20260812 19:02:48.311110 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 17595
2026-08-12T19:02:48Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59318 ms)
[----------] 30 tests from SecurityITest (243291 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:02:48 dist-test-slave-6nmv krb5kdc[17819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:02:48 dist-test-slave-6nmv krb5kdc[17819](info): set up 2 sockets
Aug 12 19:02:48 dist-test-slave-6nmv krb5kdc[17819](info): commencing operation
krb5kdc: starting...
W20260812 19:02:49.436681 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:02:49 dist-test-slave-6nmv krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:02:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:02:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:02:49.564499 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:46625
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:46625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:49.675287 17837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:49.675628 17837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:49.675714 17837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:49.679728 17837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:02:49.679826 17837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:49.679867 17837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:49.679904 17837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:02:49.679937 17837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:02:49.685194 17837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:46625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:46625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:49.686308 17837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:49.687335 17837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:49.695894 17842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:49.695858 17845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:49.696012 17837 server_base.cc:1034] running on GCE node
W20260812 19:02:49.696221 17843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:49.696784 17837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:49.697911 17837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:49.699112 17837 hybrid_clock.cc:648] HybridClock initialized: now 1786561369699102 us; error 26 us; skew 500 ppm
Aug 12 19:02:49 dist-test-slave-6nmv krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561369, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:49.707049 17837 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:02:49.707929 17837 webserver.cc:460] Webserver started at http://127.29.112.254:34799/ using document root <none> and password file <none>
I20260812 19:02:49.708509 17837 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:49.708602 17837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:49.708844 17837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:49.710933 17837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "751019d1813b4d309d9282437a86ff43"
format_stamp: "Formatted at 2026-08-12 19:02:49 on dist-test-slave-6nmv"
server_key: "07a83332fd5b4a2f49d3aef2e678705e"
server_key_iv: "b9640ba076d9dde24bf63eb9f6787992"
server_key_version: "encryptionkey@0"
I20260812 19:02:49.711505 17837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "751019d1813b4d309d9282437a86ff43"
format_stamp: "Formatted at 2026-08-12 19:02:49 on dist-test-slave-6nmv"
server_key: "07a83332fd5b4a2f49d3aef2e678705e"
server_key_iv: "b9640ba076d9dde24bf63eb9f6787992"
server_key_version: "encryptionkey@0"
I20260812 19:02:49.715167 17837 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:02:49.717440 17854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:49.718719 17837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:02:49.718858 17837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "751019d1813b4d309d9282437a86ff43"
format_stamp: "Formatted at 2026-08-12 19:02:49 on dist-test-slave-6nmv"
server_key: "07a83332fd5b4a2f49d3aef2e678705e"
server_key_iv: "b9640ba076d9dde24bf63eb9f6787992"
server_key_version: "encryptionkey@0"
I20260812 19:02:49.718986 17837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:49.739163 17837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:49.742502 17837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:49.742717 17837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:49.750088 17837 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:46625
I20260812 19:02:49.750118 17916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:46625 every 8 connection(s)
I20260812 19:02:49.751032 17837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:02:49.753716 17917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:49.759009 17917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43: Bootstrap starting.
I20260812 19:02:49.761082 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 17837
I20260812 19:02:49.761205 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:02:49.761521 17917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:49.761569 30147 external_mini_cluster.cc:1428] Setting key 2d821918d771600563f984d8cc525a74
I20260812 19:02:49.762384 17917 log.cc:824] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:49.764590 17917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43: No bootstrap required, opened a new log
I20260812 19:02:49.768692 17917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "751019d1813b4d309d9282437a86ff43" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46625 } }
I20260812 19:02:49.768950 17917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:49.769048 17917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 751019d1813b4d309d9282437a86ff43, State: Initialized, Role: FOLLOWER
I20260812 19:02:49.769510 17917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "751019d1813b4d309d9282437a86ff43" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46625 } }
I20260812 19:02:49.769632 17917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:02:49.769711 17917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:02:49.769841 17917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:49.770759 17917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "751019d1813b4d309d9282437a86ff43" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46625 } }
Aug 12 19:02:49 dist-test-slave-6nmv krb5kdc[17819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:49.771140 17917 leader_election.cc:302] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 751019d1813b4d309d9282437a86ff43; no voters:
I20260812 19:02:49.771478 17917 leader_election.cc:288] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:02:49.771597 17924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:49.771858 17924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [term 1 LEADER]: Becoming Leader. State: Replica: 751019d1813b4d309d9282437a86ff43, State: Running, Role: LEADER
I20260812 19:02:49.772246 17924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "751019d1813b4d309d9282437a86ff43" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46625 } }
I20260812 19:02:49.772815 17917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:02:49.774060 17926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 751019d1813b4d309d9282437a86ff43. Latest consensus state: current_term: 1 leader_uuid: "751019d1813b4d309d9282437a86ff43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "751019d1813b4d309d9282437a86ff43" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46625 } } }
I20260812 19:02:49.774192 17926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:49.774544 17925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "751019d1813b4d309d9282437a86ff43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "751019d1813b4d309d9282437a86ff43" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 46625 } } }
I20260812 19:02:49.774627 17935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:02:49.774649 17925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:49.777777 17935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:02:49.778134 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:49.762828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.29.112.254:46625)
0812 19:02:49.763302 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:49.763324 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:49.763365 (+ 41us) server_negotiation.cc:407] Connection header received
0812 19:02:49.764188 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:49.764209 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:49.764576 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:49.764924 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:49.765599 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:49.767163 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:49.768075 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:49.768712 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:49.768862 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:49.771909 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:49.771915 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:49.771931 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:49.771960 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:49.774146 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:49.774698 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:49.774705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:49.774714 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:49.774897 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:49.775279 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:49.775284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:49.775287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:49.775830 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:49.776047 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:49.776925 (+ 878us) server_negotiation.cc:299] Negotiation successful
0812 19:02:49.777205 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":161,"threads_started":1}
I20260812 19:02:49.783736 17935 catalog_manager.cc:1347] Generated new cluster ID: 76ce3b2a6ccc4502af8f05fb255ec868
I20260812 19:02:49.783934 17935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:02:49.799075 17935 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:02:49.800246 17935 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:02:49.809038 17935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43: Generated new TSK 0
I20260812 19:02:49.809705 17935 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 19:02:49.845152 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46625
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:49.954113 17947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:49.954370 17947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:49.954420 17947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:49.958389 17947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:49.958444 17947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:49.958500 17947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:02:49.963784 17947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:49.964823 17947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:49.965845 17947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:49.976325 17952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:49.976356 17955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:49.976354 17953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:49.976557 17947 server_base.cc:1034] running on GCE node
I20260812 19:02:49.976948 17947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:49.977612 17947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:49.978806 17947 hybrid_clock.cc:648] HybridClock initialized: now 1786561369978792 us; error 38 us; skew 500 ppm
Aug 12 19:02:49 dist-test-slave-6nmv krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561369, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:49.981954 17947 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:02:49.982869 17947 webserver.cc:460] Webserver started at http://127.29.112.193:33965/ using document root <none> and password file <none>
I20260812 19:02:49.983459 17947 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:49.983567 17947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:49.983801 17947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:49.985802 17947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "d82a78c30a9b42dfa015ecf2de334c30"
format_stamp: "Formatted at 2026-08-12 19:02:49 on dist-test-slave-6nmv"
server_key: "1605e73c9d53ab15d8b5cb659ff129ab"
server_key_iv: "eb24029afb5514105dccc9d85cdee097"
server_key_version: "encryptionkey@0"
I20260812 19:02:49.986449 17947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "d82a78c30a9b42dfa015ecf2de334c30"
format_stamp: "Formatted at 2026-08-12 19:02:49 on dist-test-slave-6nmv"
server_key: "1605e73c9d53ab15d8b5cb659ff129ab"
server_key_iv: "eb24029afb5514105dccc9d85cdee097"
server_key_version: "encryptionkey@0"
I20260812 19:02:49.990293 17947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:02:49.992996 17964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:49.994397 17947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:02:49.994560 17947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "d82a78c30a9b42dfa015ecf2de334c30"
format_stamp: "Formatted at 2026-08-12 19:02:49 on dist-test-slave-6nmv"
server_key: "1605e73c9d53ab15d8b5cb659ff129ab"
server_key_iv: "eb24029afb5514105dccc9d85cdee097"
server_key_version: "encryptionkey@0"
I20260812 19:02:49.994660 17947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:50.014732 17947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:50.018491 17947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:50.018743 17947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:50.019493 17947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:50.020393 17947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:50.020448 17947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:50.020498 17947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:50.020515 17947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:50.030805 17947 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:35241
I20260812 19:02:50.030826 18093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:35241 every 8 connection(s)
I20260812 19:02:50.032279 17947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:02:50.041649 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 17947
I20260812 19:02:50.041826 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 19:02:50.042151 30147 external_mini_cluster.cc:1428] Setting key 3c2fcd16b779813ff29fe14fb5db0381
Aug 12 19:02:50 dist-test-slave-6nmv krb5kdc[17819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561369, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:50.047343 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.034297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:35155 (local address 127.29.112.254:46625)
0812 19:02:50.034476 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:50.034482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:50.035332 (+ 850us) server_negotiation.cc:407] Connection header received
0812 19:02:50.036710 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:50.036713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:50.036760 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:50.036838 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:50.038145 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.038703 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.039871 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.040130 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.040217 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.043137 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:50.043140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:50.043142 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:50.043208 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:50.044980 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:50.045576 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:50.045581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:50.045583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:50.045682 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:50.046112 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:50.046116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:50.046118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:50.046330 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:50.046449 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:50.046998 (+ 549us) server_negotiation.cc:299] Negotiation successful
0812 19:02:50.047145 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 19:02:50.048182 18097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.034686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:46625 (local address 127.29.112.193:35155)
0812 19:02:50.035165 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:50.035209 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:50.036494 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:50.037024 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:50.037031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:50.037399 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:50.037991 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.038009 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.038838 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.038841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.039747 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.039758 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.040292 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.040295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.040451 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.041110 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:50.041127 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:50.042924 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:50.045130 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:50.045136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:50.045152 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:50.045456 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:50.045795 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:50.045799 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:50.045801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:50.045992 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:50.046462 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:50.046470 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:50.046735 (+ 265us) client_negotiation.cc:769] Sending connection context
0812 19:02:50.046938 (+ 203us) client_negotiation.cc:240] Negotiation successful
0812 19:02:50.047293 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":135,"threads_started":1}
I20260812 19:02:50.049531 18094 heartbeater.cc:347] Connected to a master server at 127.29.112.254:46625
I20260812 19:02:50.049801 18094 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:50.050415 18094 heartbeater.cc:511] Master 127.29.112.254:46625 requested a full tablet report, sending...
I20260812 19:02:50.052264 17876 ts_manager.cc:194] Registered new tserver with Master: d82a78c30a9b42dfa015ecf2de334c30 (127.29.112.193:35241)
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 19:02:50.053592 17876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:35155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 19:02:50.073256 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46625
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:50.185415 18100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:50.185647 18100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:50.185686 18100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:50.189636 18100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:50.189684 18100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:50.189739 18100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:02:50.194990 18100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:50.195843 18100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:50.196811 18100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:50.206171 18108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:50.206144 18112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:50.206135 18107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:50.206332 18100 server_base.cc:1034] running on GCE node
I20260812 19:02:50.206923 18100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:50.207523 18100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:50.208719 18100 hybrid_clock.cc:648] HybridClock initialized: now 1786561370208688 us; error 46 us; skew 500 ppm
Aug 12 19:02:50 dist-test-slave-6nmv krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561370, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:50.211494 18100 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:02:50.212323 18100 webserver.cc:460] Webserver started at http://127.29.112.194:35943/ using document root <none> and password file <none>
I20260812 19:02:50.212872 18100 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:50.212983 18100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:50.213214 18100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:50.215245 18100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "b3e1791391bc469c9fff606690d3a694"
format_stamp: "Formatted at 2026-08-12 19:02:50 on dist-test-slave-6nmv"
server_key: "fb19a13f787207e9721ddd27268a531a"
server_key_iv: "ea88e420163e8860fbf8dcc2129d2e91"
server_key_version: "encryptionkey@0"
I20260812 19:02:50.215817 18100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "b3e1791391bc469c9fff606690d3a694"
format_stamp: "Formatted at 2026-08-12 19:02:50 on dist-test-slave-6nmv"
server_key: "fb19a13f787207e9721ddd27268a531a"
server_key_iv: "ea88e420163e8860fbf8dcc2129d2e91"
server_key_version: "encryptionkey@0"
I20260812 19:02:50.219465 18100 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260812 19:02:50.221892 18119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:50.223225 18100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:02:50.223361 18100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "b3e1791391bc469c9fff606690d3a694"
format_stamp: "Formatted at 2026-08-12 19:02:50 on dist-test-slave-6nmv"
server_key: "fb19a13f787207e9721ddd27268a531a"
server_key_iv: "ea88e420163e8860fbf8dcc2129d2e91"
server_key_version: "encryptionkey@0"
I20260812 19:02:50.223474 18100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:50.238977 18100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:50.242522 18100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:50.242743 18100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:50.243376 18100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:50.244235 18100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:50.244318 18100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:50.244392 18100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:50.244429 18100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:50.253499 18100 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:45193
I20260812 19:02:50.253578 18256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:45193 every 8 connection(s)
I20260812 19:02:50.254506 18100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:02:50.260613 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 18100
I20260812 19:02:50.260721 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 19:02:50.261080 30147 external_mini_cluster.cc:1428] Setting key d1338b1552582dc35837f70d0ca07930
Aug 12 19:02:50 dist-test-slave-6nmv krb5kdc[17819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561370, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:50.269919 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.256415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:51377 (local address 127.29.112.254:46625)
0812 19:02:50.256594 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:50.256598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:50.257561 (+ 963us) server_negotiation.cc:407] Connection header received
0812 19:02:50.258989 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:50.258993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:50.259044 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:50.259126 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:50.260307 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.260936 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.262452 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.262773 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.262858 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.266041 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:50.266048 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:50.266053 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:50.266087 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:50.267678 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:50.268247 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:50.268251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:50.268252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:50.268340 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:50.268726 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:50.268729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:50.268731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:50.268930 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:50.269048 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:50.269577 (+ 529us) server_negotiation.cc:299] Negotiation successful
0812 19:02:50.269710 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 19:02:50.270740 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.256909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:46625 (local address 127.29.112.194:51377)
0812 19:02:50.257392 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:50.257447 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:50.258771 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:50.259285 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:50.259291 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:50.259666 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:50.260149 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.260162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.261088 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.261095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.262322 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.262333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.262912 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.262916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.263120 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.263778 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:50.263804 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:50.265777 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:50.267820 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:50.267827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:50.267842 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:50.268122 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:50.268443 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:50.268446 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:50.268448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:50.268619 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:50.269048 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:50.269056 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:50.269307 (+ 251us) client_negotiation.cc:769] Sending connection context
0812 19:02:50.269521 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 19:02:50.269897 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
I20260812 19:02:50.271986 18257 heartbeater.cc:347] Connected to a master server at 127.29.112.254:46625
I20260812 19:02:50.272199 18257 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:50.272683 18257 heartbeater.cc:511] Master 127.29.112.254:46625 requested a full tablet report, sending...
I20260812 19:02:50.273944 17876 ts_manager.cc:194] Registered new tserver with Master: b3e1791391bc469c9fff606690d3a694 (127.29.112.194:45193)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
I20260812 19:02:50.274760 17876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:51377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 19:02:50.291154 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46625
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:37261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:50.399303 18264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:50.399549 18264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:50.399588 18264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:50.403437 18264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:50.403486 18264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:50.403573 18264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:02:50.408598 18264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:37261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:46625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:50.409530 18264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:50.410581 18264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:50.420152 18274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:50.420153 18272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:50.420291 18271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:50.420677 18264 server_base.cc:1034] running on GCE node
I20260812 19:02:50.421056 18264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:50.421614 18264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:50.422801 18264 hybrid_clock.cc:648] HybridClock initialized: now 1786561370422791 us; error 68 us; skew 500 ppm
Aug 12 19:02:50 dist-test-slave-6nmv krb5kdc[17819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561370, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:50.425552 18264 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:02:50.426467 18264 webserver.cc:460] Webserver started at http://127.29.112.195:37563/ using document root <none> and password file <none>
I20260812 19:02:50.427039 18264 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:50.427136 18264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:50.427398 18264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:50.429343 18264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "5761804a70a54a0cb47c0f90c08838ee"
format_stamp: "Formatted at 2026-08-12 19:02:50 on dist-test-slave-6nmv"
server_key: "d097877b9e04dc8ffb91b4ff4af1a7aa"
server_key_iv: "e16a8a56dedc6dc181f869ea2d66ca19"
server_key_version: "encryptionkey@0"
I20260812 19:02:50.429986 18264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "5761804a70a54a0cb47c0f90c08838ee"
format_stamp: "Formatted at 2026-08-12 19:02:50 on dist-test-slave-6nmv"
server_key: "d097877b9e04dc8ffb91b4ff4af1a7aa"
server_key_iv: "e16a8a56dedc6dc181f869ea2d66ca19"
server_key_version: "encryptionkey@0"
I20260812 19:02:50.433496 18264 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:02:50.435859 18282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:50.437084 18264 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:02:50.437232 18264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "5761804a70a54a0cb47c0f90c08838ee"
format_stamp: "Formatted at 2026-08-12 19:02:50 on dist-test-slave-6nmv"
server_key: "d097877b9e04dc8ffb91b4ff4af1a7aa"
server_key_iv: "e16a8a56dedc6dc181f869ea2d66ca19"
server_key_version: "encryptionkey@0"
I20260812 19:02:50.437341 18264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:50.472122 18264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:50.475446 18264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:50.475651 18264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:50.476261 18264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:50.477098 18264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:50.477169 18264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:50.477237 18264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:50.477290 18264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:50.486297 18264 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:41551
I20260812 19:02:50.486351 18420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:41551 every 8 connection(s)
I20260812 19:02:50.487205 18264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:02:50.488399 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 18264
I20260812 19:02:50.488543 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 19:02:50.488883 30147 external_mini_cluster.cc:1428] Setting key fabdad51b42ef6a5d1bb9ed560db8d80
Aug 12 19:02:50 dist-test-slave-6nmv krb5kdc[17819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561370, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:50.502660 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.490005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:44349 (local address 127.29.112.254:46625)
0812 19:02:50.490141 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:50.490146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:50.490998 (+ 852us) server_negotiation.cc:407] Connection header received
0812 19:02:50.492442 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:50.492444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:50.492490 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:50.492556 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:50.493717 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.494309 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.495560 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.495851 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.495953 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.498420 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:50.498424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:50.498426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:50.498451 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:50.500155 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:50.500738 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:50.500742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:50.500744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:50.500849 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:50.501222 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:50.501225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:50.501227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:50.501477 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:50.501590 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:50.502271 (+ 681us) server_negotiation.cc:299] Negotiation successful
0812 19:02:50.502452 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 19:02:50.503593 18423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.489681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:46625 (local address 127.29.112.195:44349)
0812 19:02:50.490837 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:50.490890 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:50.492218 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:50.492705 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:50.492712 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:50.493048 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:50.493594 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.493608 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.494419 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.494421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.495303 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.495312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.496012 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.496014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.496142 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.496670 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:50.496684 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:50.498229 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:50.500305 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:50.500311 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:50.500324 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:50.500613 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:50.500949 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:50.500954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:50.500956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:50.501106 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:50.501654 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:50.501661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:50.502000 (+ 339us) client_negotiation.cc:769] Sending connection context
0812 19:02:50.502222 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 19:02:50.502571 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"thread_start_us":162,"threads_started":1}
I20260812 19:02:50.504874 18421 heartbeater.cc:347] Connected to a master server at 127.29.112.254:46625
I20260812 19:02:50.505086 18421 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:50.505613 18421 heartbeater.cc:511] Master 127.29.112.254:46625 requested a full tablet report, sending...
I20260812 19:02:50.506625 17876 ts_manager.cc:194] Registered new tserver with Master: 5761804a70a54a0cb47c0f90c08838ee (127.29.112.195:41551)
I20260812 19:02:50.507201 17876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:44349
I20260812 19:02:50.514930 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:02:50.524065 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.517055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40768 (local address 127.29.112.254:46625)
0812 19:02:50.517222 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:50.517225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:50.517318 (+ 93us) server_negotiation.cc:407] Connection header received
0812 19:02:50.517439 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:50.517441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:50.517489 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:50.517563 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:50.518335 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.518775 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.519531 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.519710 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.519781 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.521149 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:50.521151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:50.521153 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:50.521169 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:50.522462 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:50.522866 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:50.522869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:50.522870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:50.522987 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:50.523225 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:50.523226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:50.523227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:50.523425 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:50.523535 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:50.523740 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 19:02:50.523862 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 19:02:50.526814 17876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:02:50.529677 17876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:02:50.543128 18438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.537721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.29.112.195:41551)
0812 19:02:50.538080 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:50.538090 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:50.538330 (+ 240us) server_negotiation.cc:407] Connection header received
0812 19:02:50.538559 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:50.538565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:50.538779 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:50.538901 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:50.539368 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.540026 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.541428 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.542283 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.542391 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.542502 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:50.542862 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 19:02:50.542980 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":190,"threads_started":1}
I20260812 19:02:50.543207 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.538017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:45193 (local address 127.0.0.1:51492)
0812 19:02:50.538733 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:50.538746 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:50.538824 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:50.539294 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:50.539297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:50.539314 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:50.539412 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.539418 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.540695 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.540698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.542003 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.542014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.542781 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.542784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.542869 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.542888 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 19:02:50.542974 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 19:02:50.543095 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":110,"thread_start_us":97,"threads_started":1}
I20260812 19:02:50.543435 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.538156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51492 (local address 127.29.112.194:45193)
0812 19:02:50.538525 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:50.538536 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:50.538902 (+ 366us) server_negotiation.cc:407] Connection header received
0812 19:02:50.538972 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:50.538977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:50.539125 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:50.539539 (+ 414us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:50.539675 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.540595 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.542155 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.542658 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.542790 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.542906 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:50.542998 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 19:02:50.543091 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":110,"threads_started":1}
I20260812 19:02:50.543453 18435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.537338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:35241 (local address 127.0.0.1:37996)
0812 19:02:50.537808 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:50.537881 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:50.538437 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:50.539100 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:50.539102 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:50.539127 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:50.539246 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.539253 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.540666 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.540669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.542003 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.542014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.543033 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.543036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.543131 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.543158 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 19:02:50.543209 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 19:02:50.543309 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":108,"spinlock_wait_cycles":768,"thread_start_us":77,"threads_started":1}
I20260812 19:02:50.543608 18440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.537576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:41551 (local address 127.0.0.1:41738)
0812 19:02:50.538228 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:50.538238 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:50.538377 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:50.539086 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:50.539093 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:50.539123 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:50.539250 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.539257 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.540167 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.540170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.541211 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.541222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.542560 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.542563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.542672 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.542707 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 19:02:50.543426 (+ 719us) client_negotiation.cc:240] Negotiation successful
0812 19:02:50.543494 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":1,"spinlock_wait_cycles":2304,"thread_start_us":89,"threads_started":1}
I20260812 19:02:50.544001 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.538300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37996 (local address 127.29.112.193:35241)
0812 19:02:50.538676 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:50.538694 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:50.538714 (+ 20us) server_negotiation.cc:407] Connection header received
0812 19:02:50.538789 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:50.538795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:50.538967 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:50.539381 (+ 414us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:50.539518 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.540536 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.542149 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.542913 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.543612 (+ 699us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.543753 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:50.543799 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 19:02:50.543878 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":94,"threads_started":1}
I20260812 19:02:50.546005 18342 tablet_service.cc:1467] Processing CreateTablet for tablet 90ff6a6dc63648c192172d42df1f9315 (DEFAULT_TABLE table=test-table [id=8db4a7f41bed4815a4a99d1660bfa7a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:50.546108 18019 tablet_service.cc:1467] Processing CreateTablet for tablet 90ff6a6dc63648c192172d42df1f9315 (DEFAULT_TABLE table=test-table [id=8db4a7f41bed4815a4a99d1660bfa7a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:50.546105 18179 tablet_service.cc:1467] Processing CreateTablet for tablet 90ff6a6dc63648c192172d42df1f9315 (DEFAULT_TABLE table=test-table [id=8db4a7f41bed4815a4a99d1660bfa7a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:50.547020 18342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90ff6a6dc63648c192172d42df1f9315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:50.547057 18179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90ff6a6dc63648c192172d42df1f9315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:50.547057 18019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90ff6a6dc63648c192172d42df1f9315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:50.552678 18443 tablet_bootstrap.cc:492] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: Bootstrap starting.
I20260812 19:02:50.554261 18445 tablet_bootstrap.cc:492] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694: Bootstrap starting.
I20260812 19:02:50.555009 18443 tablet_bootstrap.cc:654] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:50.556354 18443 log.cc:824] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:50.556442 18445 tablet_bootstrap.cc:654] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:50.556969 18444 tablet_bootstrap.cc:492] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30: Bootstrap starting.
I20260812 19:02:50.557658 18445 log.cc:824] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:50.558238 18443 tablet_bootstrap.cc:492] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: No bootstrap required, opened a new log
I20260812 19:02:50.558449 18443 ts_tablet_manager.cc:1397] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 19:02:50.558945 18444 tablet_bootstrap.cc:654] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:50.559600 18445 tablet_bootstrap.cc:492] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694: No bootstrap required, opened a new log
I20260812 19:02:50.559741 18444 log.cc:824] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:50.559793 18445 ts_tablet_manager.cc:1397] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 19:02:50.561546 18444 tablet_bootstrap.cc:492] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30: No bootstrap required, opened a new log
I20260812 19:02:50.561725 18444 ts_tablet_manager.cc:1397] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260812 19:02:50.561813 18443 raft_consensus.cc:348] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82a78c30a9b42dfa015ecf2de334c30" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 35241 } } peers { permanent_uuid: "5761804a70a54a0cb47c0f90c08838ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41551 } } peers { permanent_uuid: "b3e1791391bc469c9fff606690d3a694" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45193 } }
I20260812 19:02:50.562071 18443 raft_consensus.cc:374] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:50.562160 18443 raft_consensus.cc:729] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5761804a70a54a0cb47c0f90c08838ee, State: Initialized, Role: FOLLOWER
I20260812 19:02:50.562683 18443 consensus_queue.cc:260] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82a78c30a9b42dfa015ecf2de334c30" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 35241 } } peers { permanent_uuid: "5761804a70a54a0cb47c0f90c08838ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41551 } } peers { permanent_uuid: "b3e1791391bc469c9fff606690d3a694" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45193 } }
I20260812 19:02:50.563267 18421 heartbeater.cc:503] Master 127.29.112.254:46625 was elected leader, sending a full tablet report...
I20260812 19:02:50.563603 18443 ts_tablet_manager.cc:1428] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 19:02:50.563668 18445 raft_consensus.cc:348] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82a78c30a9b42dfa015ecf2de334c30" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 35241 } } peers { permanent_uuid: "5761804a70a54a0cb47c0f90c08838ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41551 } } peers { permanent_uuid: "b3e1791391bc469c9fff606690d3a694" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45193 } }
I20260812 19:02:50.563870 18445 raft_consensus.cc:374] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:50.563925 18445 raft_consensus.cc:729] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3e1791391bc469c9fff606690d3a694, State: Initialized, Role: FOLLOWER
I20260812 19:02:50.564332 18445 consensus_queue.cc:260] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82a78c30a9b42dfa015ecf2de334c30" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 35241 } } peers { permanent_uuid: "5761804a70a54a0cb47c0f90c08838ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41551 } } peers { permanent_uuid: "b3e1791391bc469c9fff606690d3a694" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45193 } }
I20260812 19:02:50.564921 18257 heartbeater.cc:503] Master 127.29.112.254:46625 was elected leader, sending a full tablet report...
I20260812 19:02:50.565198 18445 ts_tablet_manager.cc:1428] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260812 19:02:50.565965 18444 raft_consensus.cc:348] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82a78c30a9b42dfa015ecf2de334c30" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 35241 } } peers { permanent_uuid: "5761804a70a54a0cb47c0f90c08838ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41551 } } peers { permanent_uuid: "b3e1791391bc469c9fff606690d3a694" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45193 } }
I20260812 19:02:50.566248 18444 raft_consensus.cc:374] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:50.566316 18444 raft_consensus.cc:729] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d82a78c30a9b42dfa015ecf2de334c30, State: Initialized, Role: FOLLOWER
I20260812 19:02:50.566741 18444 consensus_queue.cc:260] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82a78c30a9b42dfa015ecf2de334c30" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 35241 } } peers { permanent_uuid: "5761804a70a54a0cb47c0f90c08838ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41551 } } peers { permanent_uuid: "b3e1791391bc469c9fff606690d3a694" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45193 } }
I20260812 19:02:50.567682 18094 heartbeater.cc:503] Master 127.29.112.254:46625 was elected leader, sending a full tablet report...
I20260812 19:02:50.567957 18444 ts_tablet_manager.cc:1428] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 19:02:50.684247 18449 raft_consensus.cc:482] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:50.684535 18449 raft_consensus.cc:504] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82a78c30a9b42dfa015ecf2de334c30" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 35241 } } peers { permanent_uuid: "5761804a70a54a0cb47c0f90c08838ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41551 } } peers { permanent_uuid: "b3e1791391bc469c9fff606690d3a694" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45193 } }
I20260812 19:02:50.685658 18449 leader_election.cc:288] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d82a78c30a9b42dfa015ecf2de334c30 (127.29.112.193:35241), b3e1791391bc469c9fff606690d3a694 (127.29.112.194:45193)
I20260812 19:02:50.690060 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.686161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:33027 (local address 127.29.112.193:35241)
0812 19:02:50.686324 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:50.686330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:50.686352 (+ 22us) server_negotiation.cc:407] Connection header received
0812 19:02:50.686432 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:50.686435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:50.686492 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:50.686585 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:50.687008 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.687738 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.688955 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.689539 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.689630 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.689671 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:50.689908 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 19:02:50.689964 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:02:50.690125 18423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.686067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:35241 (local address 127.29.112.195:33027)
0812 19:02:50.686198 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:50.686214 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:50.686315 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:50.686697 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:50.686700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:50.686724 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:50.686844 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.686856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.687854 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.687858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.688819 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.688828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.689655 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.689658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.689762 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.689788 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 19:02:50.689892 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 19:02:50.689985 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 19:02:50.690620 18452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.686374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:45193 (local address 127.29.112.195:33151)
0812 19:02:50.686819 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:50.686832 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:50.686944 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:50.687324 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:50.687327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:50.687348 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:50.687474 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.687479 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.688366 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.688369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.689302 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:50.689313 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.690156 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.690160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:50.690280 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.690315 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 19:02:50.690406 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 19:02:50.690474 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":185,"threads_started":1}
I20260812 19:02:50.690637 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.686500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:33151 (local address 127.29.112.194:45193)
0812 19:02:50.686625 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:50.686629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:50.686910 (+ 281us) server_negotiation.cc:407] Connection header received
0812 19:02:50.687069 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:50.687072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:50.687126 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:50.687195 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:50.687579 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.688254 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.689439 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.690034 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.690148 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.690202 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:50.690434 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 19:02:50.690521 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 19:02:50.690845 18042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ff6a6dc63648c192172d42df1f9315" candidate_uuid: "5761804a70a54a0cb47c0f90c08838ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d82a78c30a9b42dfa015ecf2de334c30" is_pre_election: true
I20260812 19:02:50.691183 18042 raft_consensus.cc:2454] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5761804a70a54a0cb47c0f90c08838ee in term 0.
I20260812 19:02:50.691357 18201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ff6a6dc63648c192172d42df1f9315" candidate_uuid: "5761804a70a54a0cb47c0f90c08838ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3e1791391bc469c9fff606690d3a694" is_pre_election: true
I20260812 19:02:50.691648 18201 raft_consensus.cc:2454] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5761804a70a54a0cb47c0f90c08838ee in term 0.
I20260812 19:02:50.691733 18286 leader_election.cc:302] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5761804a70a54a0cb47c0f90c08838ee, d82a78c30a9b42dfa015ecf2de334c30; no voters:
I20260812 19:02:50.692059 18449 raft_consensus.cc:2781] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:02:50.692133 18449 raft_consensus.cc:482] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:02:50.692163 18449 raft_consensus.cc:3037] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:50.693222 18449 raft_consensus.cc:504] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82a78c30a9b42dfa015ecf2de334c30" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 35241 } } peers { permanent_uuid: "5761804a70a54a0cb47c0f90c08838ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41551 } } peers { permanent_uuid: "b3e1791391bc469c9fff606690d3a694" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45193 } }
I20260812 19:02:50.693591 18449 leader_election.cc:288] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [CANDIDATE]: Term 1 election: Requested vote from peers d82a78c30a9b42dfa015ecf2de334c30 (127.29.112.193:35241), b3e1791391bc469c9fff606690d3a694 (127.29.112.194:45193)
I20260812 19:02:50.694051 18201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ff6a6dc63648c192172d42df1f9315" candidate_uuid: "5761804a70a54a0cb47c0f90c08838ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3e1791391bc469c9fff606690d3a694"
I20260812 19:02:50.694059 18042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ff6a6dc63648c192172d42df1f9315" candidate_uuid: "5761804a70a54a0cb47c0f90c08838ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d82a78c30a9b42dfa015ecf2de334c30"
I20260812 19:02:50.694175 18042 raft_consensus.cc:3037] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:50.694175 18201 raft_consensus.cc:3037] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:50.695556 18201 raft_consensus.cc:2454] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5761804a70a54a0cb47c0f90c08838ee in term 1.
I20260812 19:02:50.695556 18042 raft_consensus.cc:2454] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5761804a70a54a0cb47c0f90c08838ee in term 1.
I20260812 19:02:50.695840 18286 leader_election.cc:302] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5761804a70a54a0cb47c0f90c08838ee, b3e1791391bc469c9fff606690d3a694; no voters:
I20260812 19:02:50.696034 18449 raft_consensus.cc:2781] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:50.696275 18449 raft_consensus.cc:686] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 1 LEADER]: Becoming Leader. State: Replica: 5761804a70a54a0cb47c0f90c08838ee, State: Running, Role: LEADER
I20260812 19:02:50.696633 18449 consensus_queue.cc:237] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82a78c30a9b42dfa015ecf2de334c30" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 35241 } } peers { permanent_uuid: "5761804a70a54a0cb47c0f90c08838ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41551 } } peers { permanent_uuid: "b3e1791391bc469c9fff606690d3a694" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45193 } }
I20260812 19:02:50.699810 17876 catalog_manager.cc:5495] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 5761804a70a54a0cb47c0f90c08838ee (127.29.112.195). New cstate: current_term: 1 leader_uuid: "5761804a70a54a0cb47c0f90c08838ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d82a78c30a9b42dfa015ecf2de334c30" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 35241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5761804a70a54a0cb47c0f90c08838ee" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3e1791391bc469c9fff606690d3a694" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45193 } health_report { overall_health: UNKNOWN } } }
I20260812 19:02:50.732285 18438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.729013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41744 (local address 127.29.112.195:41551)
0812 19:02:50.729144 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:50.729148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:50.729163 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:02:50.729252 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:50.729254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:50.729307 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:50.729393 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:02:50.729619 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.730131 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.730924 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:50.731125 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:50.731200 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:50.731466 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:02:50.731877 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:02:50.731969 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:50.732026 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 19:02:50.732072 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 19:02:50.738065 18422 tablet.cc:2369] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:02:50.738091 18042 raft_consensus.cc:1261] T 90ff6a6dc63648c192172d42df1f9315 P d82a78c30a9b42dfa015ecf2de334c30 [term 1 FOLLOWER]: Refusing update from remote peer 5761804a70a54a0cb47c0f90c08838ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:50.738091 18201 raft_consensus.cc:1261] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694 [term 1 FOLLOWER]: Refusing update from remote peer 5761804a70a54a0cb47c0f90c08838ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:50.738790 18453 consensus_queue.cc:1035] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d82a78c30a9b42dfa015ecf2de334c30" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 35241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:50.739041 18449 consensus_queue.cc:1035] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3e1791391bc469c9fff606690d3a694" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:50.747468 18457 mvcc.cc:205] Tried to move back new op lower bound from 7317755374535618560 to 7317755374376181760. Current Snapshot: MvccSnapshot[applied={T|T < 7317755374535618560}]
I20260812 19:02:50.748260 18458 mvcc.cc:205] Tried to move back new op lower bound from 7317755374535618560 to 7317755374376181760. Current Snapshot: MvccSnapshot[applied={T|T < 7317755374535618560}]
I20260812 19:02:50.755175 17873 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:50.755342 17873 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:50.758019 17873 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 751019d1813b4d309d9282437a86ff43: Sending DeleteTablet for 3 replicas of tablet 90ff6a6dc63648c192172d42df1f9315
I20260812 19:02:50.758801 18342 tablet_service.cc:1514] Processing DeleteTablet for tablet 90ff6a6dc63648c192172d42df1f9315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:50 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:41738
I20260812 19:02:50.758801 18179 tablet_service.cc:1514] Processing DeleteTablet for tablet 90ff6a6dc63648c192172d42df1f9315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:50 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:51492
I20260812 19:02:50.758810 18019 tablet_service.cc:1514] Processing DeleteTablet for tablet 90ff6a6dc63648c192172d42df1f9315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:50 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:37996
I20260812 19:02:50.759891 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 17947
I20260812 19:02:50.764732 18467 tablet_replica.cc:330] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: stopping tablet replica
I20260812 19:02:50.765082 18467 raft_consensus.cc:2229] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:02:50.765589 18467 raft_consensus.cc:2258] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:50.766021 18466 tablet_replica.cc:330] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694: stopping tablet replica
I20260812 19:02:50.767423 18467 ts_tablet_manager.cc:1905] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:50.767797 18466 raft_consensus.cc:2229] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:02:50.768147 18466 raft_consensus.cc:2258] T 90ff6a6dc63648c192172d42df1f9315 P b3e1791391bc469c9fff606690d3a694 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:50.771035 18467 ts_tablet_manager.cc:1918] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:50.771490 18467 log.cc:1196] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/90ff6a6dc63648c192172d42df1f9315
W20260812 19:02:50.772235 17860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:35241 (error 108)
I20260812 19:02:50.772341 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 18100
I20260812 19:02:50.774685 18467 ts_tablet_manager.cc:1939] T 90ff6a6dc63648c192172d42df1f9315 P 5761804a70a54a0cb47c0f90c08838ee: Deleting consensus metadata
I20260812 19:02:50.778417 17859 catalog_manager.cc:4841] TS 5761804a70a54a0cb47c0f90c08838ee (127.29.112.195:41551): tablet 90ff6a6dc63648c192172d42df1f9315 (table test-table [id=8db4a7f41bed4815a4a99d1660bfa7a7]) successfully deleted
I20260812 19:02:50.780259 18440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.779972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:35241 (local address 127.0.0.1:38000)
0812 19:02:50.780115 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:50.780192 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:35241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 19:02:50.780478 17860 catalog_manager.cc:4584] TS d82a78c30a9b42dfa015ecf2de334c30 (127.29.112.193:35241): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90ff6a6dc63648c192172d42df1f9315: Network error: Client connection negotiation failed: client connection to 127.29.112.193:35241: connect: Connection refused (error 111)
I20260812 19:02:50.784559 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 18264
I20260812 19:02:50.785730 18440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:50.784731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:45193 (local address 127.0.0.1:51508)
0812 19:02:50.785595 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:50.785653 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.194:45193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":773}
I20260812 19:02:50.800410 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 17837
2026-08-12T19:02:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2489 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T19:02:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:02:50Z Disabled control of system clock
I20260812 19:02:50.832557 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:41141
--webserver_interface=127.29.112.254
--webserver_port=0
--builtin_ntp_servers=127.29.112.212:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:41141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:02:50.938652 18476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:50.938898 18476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:50.938933 18476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:50.944114 18476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:02:50.944175 18476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:50.944190 18476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:50.944206 18476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:02:50.944221 18476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:02:50.948948 18476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:41141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:41141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:50.950055 18476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:50.951339 18476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:02:50.959746 18476 server_base.cc:1034] running on GCE node
W20260812 19:02:50.959714 18487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:50.959714 18483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:50.959728 18484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:50.960489 18476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:50.961489 18476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:50.962652 18476 hybrid_clock.cc:648] HybridClock initialized: now 1786561370962635 us; error 30 us; skew 500 ppm
I20260812 19:02:50.964483 18476 webserver.cc:460] Webserver started at http://127.29.112.254:45705/ using document root <none> and password file <none>
I20260812 19:02:50.965024 18476 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:50.965091 18476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:50.965324 18476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:50.967336 18476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "253577fb9a3a49558880b032c69ea7e7"
format_stamp: "Formatted at 2026-08-12 19:02:50 on dist-test-slave-6nmv"
server_key: "8423f090e9e238ecf43141c4251b30d2"
server_key_iv: "77c35556fee8670c2e8a72781a83e2ca"
server_key_version: "encryptionkey@0"
I20260812 19:02:50.967917 18476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "253577fb9a3a49558880b032c69ea7e7"
format_stamp: "Formatted at 2026-08-12 19:02:50 on dist-test-slave-6nmv"
server_key: "8423f090e9e238ecf43141c4251b30d2"
server_key_iv: "77c35556fee8670c2e8a72781a83e2ca"
server_key_version: "encryptionkey@0"
I20260812 19:02:50.971405 18476 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 19:02:50.973745 18494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:50.974895 18476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:02:50.975028 18476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "253577fb9a3a49558880b032c69ea7e7"
format_stamp: "Formatted at 2026-08-12 19:02:50 on dist-test-slave-6nmv"
server_key: "8423f090e9e238ecf43141c4251b30d2"
server_key_iv: "77c35556fee8670c2e8a72781a83e2ca"
server_key_version: "encryptionkey@0"
I20260812 19:02:50.975152 18476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:50.991528 18476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:50.992218 18476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:50.992416 18476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:50.999567 18476 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:41141
I20260812 19:02:50.999586 18559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:41141 every 8 connection(s)
I20260812 19:02:51.000491 18476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:02:51.003746 18561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:51.007930 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 18476
I20260812 19:02:51.008059 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:02:51.008347 30147 external_mini_cluster.cc:1428] Setting key ae09dabac3c812c6de1b6bee0f311af8
I20260812 19:02:51.009665 18561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7: Bootstrap starting.
I20260812 19:02:51.011881 18561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:51.012607 18561 log.cc:824] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:51.014348 18561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7: No bootstrap required, opened a new log
I20260812 19:02:51.017226 18561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253577fb9a3a49558880b032c69ea7e7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41141 } }
I20260812 19:02:51.017473 18561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:51.017554 18561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 253577fb9a3a49558880b032c69ea7e7, State: Initialized, Role: FOLLOWER
I20260812 19:02:51.017640 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.009706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58720 (local address 127.29.112.254:41141)
0812 19:02:51.010361 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:51.010374 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:51.010407 (+ 33us) server_negotiation.cc:407] Connection header received
0812 19:02:51.011001 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:51.011017 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:51.011371 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:51.011727 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:51.012155 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.013161 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.014047 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.014508 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.014673 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.015526 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:51.015533 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:51.015550 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:51.015898 (+ 348us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:51.015951 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:51.015971 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:51.016076 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:51.016187 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 19:02:51.016447 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":139,"threads_started":1}
I20260812 19:02:51.018031 18561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253577fb9a3a49558880b032c69ea7e7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41141 } }
I20260812 19:02:51.018177 18561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:02:51.018260 18561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:02:51.018348 18561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:51.019356 18561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253577fb9a3a49558880b032c69ea7e7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41141 } }
I20260812 19:02:51.019718 18561 leader_election.cc:302] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 253577fb9a3a49558880b032c69ea7e7; no voters:
I20260812 19:02:51.020084 18568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:51.020094 18561 leader_election.cc:288] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:02:51.020308 18568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [term 1 LEADER]: Becoming Leader. State: Replica: 253577fb9a3a49558880b032c69ea7e7, State: Running, Role: LEADER
I20260812 19:02:51.020576 18568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253577fb9a3a49558880b032c69ea7e7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41141 } }
I20260812 19:02:51.021231 18561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:02:51.021566 18568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "253577fb9a3a49558880b032c69ea7e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253577fb9a3a49558880b032c69ea7e7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41141 } } }
I20260812 19:02:51.021770 18568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:51.021968 18569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 253577fb9a3a49558880b032c69ea7e7. Latest consensus state: current_term: 1 leader_uuid: "253577fb9a3a49558880b032c69ea7e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253577fb9a3a49558880b032c69ea7e7" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41141 } } }
I20260812 19:02:51.022111 18569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:51.022233 18572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:02:51.025116 18572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:02:51.029946 18572 catalog_manager.cc:1347] Generated new cluster ID: f848dedb2c854d5791f30c22a2af37ac
I20260812 19:02:51.030021 18572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:02:51.049075 18572 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:02:51.050086 18572 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:02:51.061302 18572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7: Generated new TSK 0
I20260812 19:02:51.062112 18572 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:02:51.076763 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41141
--builtin_ntp_servers=127.29.112.212:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:02:51.185562 18591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:51.185787 18591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:51.185875 18591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:51.189693 18591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:51.189738 18591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:51.189788 18591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:02:51.194320 18591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:51.195326 18591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:51.196592 18591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:51.206081 18597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:51.206125 18600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:51.206190 18591 server_base.cc:1034] running on GCE node
W20260812 19:02:51.206136 18598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:51.206626 18591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:51.207233 18591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:51.208406 18591 hybrid_clock.cc:648] HybridClock initialized: now 1786561371208381 us; error 36 us; skew 500 ppm
I20260812 19:02:51.210237 18591 webserver.cc:460] Webserver started at http://127.29.112.193:40797/ using document root <none> and password file <none>
I20260812 19:02:51.210732 18591 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:51.210803 18591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:51.210953 18591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:51.212919 18591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "2e74453211d64703babfc9fc1c7caddd"
format_stamp: "Formatted at 2026-08-12 19:02:51 on dist-test-slave-6nmv"
server_key: "2e08049797ffd43fee9132e8411cd1d3"
server_key_iv: "bda18736416c3a6d462dc890b83b765c"
server_key_version: "encryptionkey@0"
I20260812 19:02:51.213493 18591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "2e74453211d64703babfc9fc1c7caddd"
format_stamp: "Formatted at 2026-08-12 19:02:51 on dist-test-slave-6nmv"
server_key: "2e08049797ffd43fee9132e8411cd1d3"
server_key_iv: "bda18736416c3a6d462dc890b83b765c"
server_key_version: "encryptionkey@0"
I20260812 19:02:51.217105 18591 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 19:02:51.219568 18606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:51.220808 18591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 19:02:51.220907 18591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "2e74453211d64703babfc9fc1c7caddd"
format_stamp: "Formatted at 2026-08-12 19:02:51 on dist-test-slave-6nmv"
server_key: "2e08049797ffd43fee9132e8411cd1d3"
server_key_iv: "bda18736416c3a6d462dc890b83b765c"
server_key_version: "encryptionkey@0"
I20260812 19:02:51.221027 18591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:51.237761 18591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:51.238667 18591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:51.238853 18591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:51.239425 18591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:51.240304 18591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:51.240351 18591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:51.240396 18591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:51.240451 18591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:51.250072 18591 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:36761
I20260812 19:02:51.250125 18743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:36761 every 8 connection(s)
I20260812 19:02:51.250926 18591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:02:51.254091 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 18591
I20260812 19:02:51.254218 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 19:02:51.254554 30147 external_mini_cluster.cc:1428] Setting key 04222ebdbdd5fe15c4bb18c26b36fbf9
I20260812 19:02:51.256955 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41141
--builtin_ntp_servers=127.29.112.212:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:02:51.261282 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.252677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:41431 (local address 127.29.112.254:41141)
0812 19:02:51.252846 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:51.252851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:51.253895 (+ 1044us) server_negotiation.cc:407] Connection header received
0812 19:02:51.255666 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:51.255670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:51.255713 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:51.255798 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:51.256981 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.257618 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.259248 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.259480 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.259597 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.260736 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:51.260740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:51.260743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:51.260765 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:51.260785 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:51.260794 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:51.260894 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:51.261065 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 19:02:51.261127 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 19:02:51.262177 18746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.253070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:41141 (local address 127.29.112.193:41431)
0812 19:02:51.253690 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:51.253741 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:51.255450 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:51.255985 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:51.255995 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:51.256102 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:51.256803 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.256827 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.257768 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.257775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.259102 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.259112 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.259633 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.259637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.259797 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.260464 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:02:51.260465 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:51.260506 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:02:51.260565 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:51.260899 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:51.260911 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:51.260920 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 19:02:51.261049 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 19:02:51.261313 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":112,"threads_started":1}
I20260812 19:02:51.263391 18744 heartbeater.cc:347] Connected to a master server at 127.29.112.254:41141
I20260812 19:02:51.263634 18744 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:51.264151 18744 heartbeater.cc:511] Master 127.29.112.254:41141 requested a full tablet report, sending...
I20260812 19:02:51.265803 18515 ts_manager.cc:194] Registered new tserver with Master: 2e74453211d64703babfc9fc1c7caddd (127.29.112.193:36761)
I20260812 19:02:51.267594 18515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.193:41431
W20260812 19:02:51.364221 18747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:51.364449 18747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:51.364483 18747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:51.368316 18747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:51.368359 18747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:51.368407 18747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:02:51.372856 18747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:51.373723 18747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:51.375005 18747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:51.384222 18757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:51.384202 18754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:51.384191 18755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:51.384375 18747 server_base.cc:1034] running on GCE node
I20260812 19:02:51.384825 18747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:51.385435 18747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:51.386618 18747 hybrid_clock.cc:648] HybridClock initialized: now 1786561371386604 us; error 26 us; skew 500 ppm
I20260812 19:02:51.388373 18747 webserver.cc:460] Webserver started at http://127.29.112.194:45863/ using document root <none> and password file <none>
I20260812 19:02:51.388921 18747 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:51.389020 18747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:51.389241 18747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:51.391270 18747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "36e09c43098e4b43b7a7ce8b37a91bd2"
format_stamp: "Formatted at 2026-08-12 19:02:51 on dist-test-slave-6nmv"
server_key: "f50902acfffc1dc50a0a80fcfe9e25df"
server_key_iv: "7d1768bf4ee95cff7f6032dfa5fb9353"
server_key_version: "encryptionkey@0"
I20260812 19:02:51.391852 18747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "36e09c43098e4b43b7a7ce8b37a91bd2"
format_stamp: "Formatted at 2026-08-12 19:02:51 on dist-test-slave-6nmv"
server_key: "f50902acfffc1dc50a0a80fcfe9e25df"
server_key_iv: "7d1768bf4ee95cff7f6032dfa5fb9353"
server_key_version: "encryptionkey@0"
I20260812 19:02:51.395504 18747 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 19:02:51.397877 18764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:51.399062 18747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:02:51.399205 18747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "36e09c43098e4b43b7a7ce8b37a91bd2"
format_stamp: "Formatted at 2026-08-12 19:02:51 on dist-test-slave-6nmv"
server_key: "f50902acfffc1dc50a0a80fcfe9e25df"
server_key_iv: "7d1768bf4ee95cff7f6032dfa5fb9353"
server_key_version: "encryptionkey@0"
I20260812 19:02:51.399309 18747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:51.423574 18747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:51.424366 18747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:51.424597 18747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:51.425264 18747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:51.426293 18747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:51.426366 18747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:51.426446 18747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:51.426484 18747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:51.435156 18747 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:34509
I20260812 19:02:51.435185 18901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:34509 every 8 connection(s)
I20260812 19:02:51.436007 18747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:02:51.443711 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 18747
I20260812 19:02:51.443861 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 19:02:51.444267 30147 external_mini_cluster.cc:1428] Setting key df232886d5d637ef2020aad6d4b40ff5
I20260812 19:02:51.445255 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.437874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:54571 (local address 127.29.112.254:41141)
0812 19:02:51.438022 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:51.438026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:51.439011 (+ 985us) server_negotiation.cc:407] Connection header received
0812 19:02:51.440372 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:51.440376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:51.440440 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:51.440522 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:51.441389 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.442001 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.443131 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.443366 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.443495 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.444739 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:51.444742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:51.444745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:51.444757 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:51.444785 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:51.444793 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:51.444878 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:51.445057 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 19:02:51.445116 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 19:02:51.446125 18904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.438233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:41141 (local address 127.29.112.194:54571)
0812 19:02:51.438856 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:51.438895 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:51.440157 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:51.440699 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:51.440705 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:51.440784 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:51.441241 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.441254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.442121 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.442127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.442999 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.443008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.443536 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.443540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.443708 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.444492 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:02:51.444493 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:51.444531 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:02:51.444590 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:51.444880 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:51.444891 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:51.444900 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 19:02:51.445050 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 19:02:51.445310 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":102,"threads_started":1}
I20260812 19:02:51.446477 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41141
--builtin_ntp_servers=127.29.112.212:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:02:51.447254 18902 heartbeater.cc:347] Connected to a master server at 127.29.112.254:41141
I20260812 19:02:51.447503 18902 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:51.448132 18902 heartbeater.cc:511] Master 127.29.112.254:41141 requested a full tablet report, sending...
I20260812 19:02:51.449229 18515 ts_manager.cc:194] Registered new tserver with Master: 36e09c43098e4b43b7a7ce8b37a91bd2 (127.29.112.194:34509)
I20260812 19:02:51.450109 18515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.194:54571
W20260812 19:02:51.560593 18905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:51.560842 18905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:51.560886 18905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:51.564986 18905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:51.565040 18905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:51.565094 18905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:02:51.569680 18905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:51.570713 18905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:51.572078 18905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:51.581460 18910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:51.581460 18913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:51.581645 18911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:51.582265 18905 server_base.cc:1034] running on GCE node
I20260812 19:02:51.582653 18905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:51.583274 18905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:51.584494 18905 hybrid_clock.cc:648] HybridClock initialized: now 1786561371584473 us; error 27 us; skew 500 ppm
I20260812 19:02:51.586319 18905 webserver.cc:460] Webserver started at http://127.29.112.195:38063/ using document root <none> and password file <none>
I20260812 19:02:51.586937 18905 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:51.587040 18905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:51.587278 18905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:51.589329 18905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "7d0a4820be364949831351cd4a50bba1"
format_stamp: "Formatted at 2026-08-12 19:02:51 on dist-test-slave-6nmv"
server_key: "af05d7af0050da6394e6fa4a9e7975db"
server_key_iv: "bc3532e182e624099b24f0b71782bde6"
server_key_version: "encryptionkey@0"
I20260812 19:02:51.589983 18905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "7d0a4820be364949831351cd4a50bba1"
format_stamp: "Formatted at 2026-08-12 19:02:51 on dist-test-slave-6nmv"
server_key: "af05d7af0050da6394e6fa4a9e7975db"
server_key_iv: "bc3532e182e624099b24f0b71782bde6"
server_key_version: "encryptionkey@0"
I20260812 19:02:51.593621 18905 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:02:51.596032 18922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:51.597226 18905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:02:51.597370 18905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "7d0a4820be364949831351cd4a50bba1"
format_stamp: "Formatted at 2026-08-12 19:02:51 on dist-test-slave-6nmv"
server_key: "af05d7af0050da6394e6fa4a9e7975db"
server_key_iv: "bc3532e182e624099b24f0b71782bde6"
server_key_version: "encryptionkey@0"
I20260812 19:02:51.597494 18905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:51.612613 18905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:51.613372 18905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:51.613588 18905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:51.614277 18905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:51.615244 18905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:51.615302 18905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:51.615345 18905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:51.615415 18905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:51.624548 18905 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:41747
I20260812 19:02:51.624591 19060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:41747 every 8 connection(s)
I20260812 19:02:51.625483 18905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:02:51.634234 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 18905
I20260812 19:02:51.634337 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 19:02:51.634681 30147 external_mini_cluster.cc:1428] Setting key 852ffd852a7af049beccd060b4535ff1
I20260812 19:02:51.635483 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.627777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:39971 (local address 127.29.112.254:41141)
0812 19:02:51.627934 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:51.627939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:51.629069 (+ 1130us) server_negotiation.cc:407] Connection header received
0812 19:02:51.630478 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:51.630480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:51.630528 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:51.630605 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:51.631492 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.632038 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.633278 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.633507 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.633587 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.634930 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:51.634933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:51.634935 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:51.634948 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:51.634971 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:51.634980 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:51.635080 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:51.635251 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 19:02:51.635315 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:02:51.636358 19063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.628196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:41141 (local address 127.29.112.195:39971)
0812 19:02:51.628915 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:51.628958 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:51.630271 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:51.630768 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:51.630774 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:51.630895 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:51.631364 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.631378 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.632262 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.632265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.633166 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.633175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.633659 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.633663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.633802 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.634638 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:02:51.634641 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:51.634693 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:02:51.634775 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:51.635085 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:51.635097 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:51.635105 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 19:02:51.635249 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 19:02:51.635517 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":117,"threads_started":1}
I20260812 19:02:51.637481 19061 heartbeater.cc:347] Connected to a master server at 127.29.112.254:41141
I20260812 19:02:51.637677 19061 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:51.638419 19061 heartbeater.cc:511] Master 127.29.112.254:41141 requested a full tablet report, sending...
I20260812 19:02:51.639420 18515 ts_manager.cc:194] Registered new tserver with Master: 7d0a4820be364949831351cd4a50bba1 (127.29.112.195:41747)
I20260812 19:02:51.640035 18515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.195:39971
I20260812 19:02:51.648664 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:02:51.654567 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.650766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58728 (local address 127.29.112.254:41141)
0812 19:02:51.650962 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:51.650967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:51.651111 (+ 144us) server_negotiation.cc:407] Connection header received
0812 19:02:51.651256 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:51.651259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:51.651302 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:51.651368 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:51.651627 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.652126 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.652911 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.653133 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.653211 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.654021 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:51.654024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:51.654027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:51.654039 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:51.654058 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:51.654075 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:51.654150 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:51.654290 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 19:02:51.654343 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 19:02:51.657630 18515 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:02:51.660770 18515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:02:51.675486 19079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.670310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.29.112.194:34509)
0812 19:02:51.670999 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:51.671012 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:51.671045 (+ 33us) server_negotiation.cc:407] Connection header received
0812 19:02:51.671119 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:51.671124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:51.671298 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:51.671721 (+ 423us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:51.671856 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.672935 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.674175 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.674724 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.674900 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.675097 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:51.675209 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 19:02:51.675307 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":134,"threads_started":1}
I20260812 19:02:51.675482 19077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.669404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:34509 (local address 127.0.0.1:53484)
0812 19:02:51.670009 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:51.670024 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:51.670112 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:51.671457 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:51.671461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:51.671478 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:51.671563 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.671569 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.673059 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.673064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.674034 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.674049 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.674889 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.674893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.675008 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.675049 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 19:02:51.675175 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 19:02:51.675299 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":53,"thread_start_us":107,"threads_started":1}
I20260812 19:02:51.675719 19075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.669188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:41747 (local address 127.0.0.1:44960)
0812 19:02:51.669557 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:51.669610 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:51.669731 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:51.670674 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:51.670680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:51.670712 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:51.670885 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.670894 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.672369 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.672373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.673934 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.673947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.675289 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.675294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.675387 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.675410 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:02:51.675485 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 19:02:51.675581 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":9,"thread_start_us":89,"threads_started":1}
I20260812 19:02:51.676198 19076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.669319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44960 (local address 127.29.112.195:41747)
0812 19:02:51.670138 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:51.670154 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:51.670182 (+ 28us) server_negotiation.cc:407] Connection header received
0812 19:02:51.670261 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:51.670267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:51.670478 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:51.670642 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:51.671038 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.672235 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.674083 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.674914 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.675043 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.675219 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:51.675910 (+ 691us) server_negotiation.cc:299] Negotiation successful
0812 19:02:51.676016 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":123,"threads_started":1}
I20260812 19:02:51.676494 19078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.669977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:36761 (local address 127.0.0.1:36730)
0812 19:02:51.670706 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:51.670719 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:51.670801 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:51.671327 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:51.671330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:51.671349 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:51.671461 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.671472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.672552 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.672556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.673940 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.673947 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.675333 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.675336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.675422 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.675450 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 19:02:51.675510 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 19:02:51.676364 (+ 854us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":886,"thread_start_us":99,"threads_started":1}
I20260812 19:02:51.676673 19081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.670433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36730 (local address 127.29.112.193:36761)
0812 19:02:51.670906 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:51.670920 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:51.670946 (+ 26us) server_negotiation.cc:407] Connection header received
0812 19:02:51.671030 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:51.671036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:51.671185 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:51.671351 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:51.671602 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.672379 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.674201 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.675043 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.675151 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.675284 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:51.676382 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0812 19:02:51.676496 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":159,"threads_started":1}
I20260812 19:02:51.678958 18986 tablet_service.cc:1467] Processing CreateTablet for tablet 59a36d1849974a6b8cf62a60bbf49ce3 (DEFAULT_TABLE table=test-table [id=7eaebd57954447beb40877b115a4fa6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:51.678951 18827 tablet_service.cc:1467] Processing CreateTablet for tablet 59a36d1849974a6b8cf62a60bbf49ce3 (DEFAULT_TABLE table=test-table [id=7eaebd57954447beb40877b115a4fa6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:51.679450 18666 tablet_service.cc:1467] Processing CreateTablet for tablet 59a36d1849974a6b8cf62a60bbf49ce3 (DEFAULT_TABLE table=test-table [id=7eaebd57954447beb40877b115a4fa6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:51.679997 18827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59a36d1849974a6b8cf62a60bbf49ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:51.680035 18986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59a36d1849974a6b8cf62a60bbf49ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:51.680377 18666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59a36d1849974a6b8cf62a60bbf49ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:51.686041 19083 tablet_bootstrap.cc:492] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2: Bootstrap starting.
I20260812 19:02:51.687929 19084 tablet_bootstrap.cc:492] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd: Bootstrap starting.
I20260812 19:02:51.688916 19083 tablet_bootstrap.cc:654] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:51.689628 19083 log.cc:824] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:51.689774 19084 tablet_bootstrap.cc:654] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:51.690165 19085 tablet_bootstrap.cc:492] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1: Bootstrap starting.
I20260812 19:02:51.690691 19084 log.cc:824] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:51.691681 19083 tablet_bootstrap.cc:492] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2: No bootstrap required, opened a new log
I20260812 19:02:51.691881 19083 ts_tablet_manager.cc:1397] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:02:51.692309 19085 tablet_bootstrap.cc:654] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:51.692831 19084 tablet_bootstrap.cc:492] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd: No bootstrap required, opened a new log
I20260812 19:02:51.693039 19084 ts_tablet_manager.cc:1397] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 19:02:51.693207 19085 log.cc:824] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:51.695211 19085 tablet_bootstrap.cc:492] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1: No bootstrap required, opened a new log
I20260812 19:02:51.695416 19085 ts_tablet_manager.cc:1397] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:02:51.695695 19083 raft_consensus.cc:348] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d0a4820be364949831351cd4a50bba1" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41747 } } peers { permanent_uuid: "2e74453211d64703babfc9fc1c7caddd" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 36761 } } peers { permanent_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 34509 } }
I20260812 19:02:51.696033 19083 raft_consensus.cc:374] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:51.696136 19083 raft_consensus.cc:729] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36e09c43098e4b43b7a7ce8b37a91bd2, State: Initialized, Role: FOLLOWER
I20260812 19:02:51.696601 19083 consensus_queue.cc:260] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d0a4820be364949831351cd4a50bba1" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41747 } } peers { permanent_uuid: "2e74453211d64703babfc9fc1c7caddd" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 36761 } } peers { permanent_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 34509 } }
I20260812 19:02:51.697229 18902 heartbeater.cc:503] Master 127.29.112.254:41141 was elected leader, sending a full tablet report...
I20260812 19:02:51.697518 19083 ts_tablet_manager.cc:1428] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:02:51.698226 19084 raft_consensus.cc:348] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d0a4820be364949831351cd4a50bba1" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41747 } } peers { permanent_uuid: "2e74453211d64703babfc9fc1c7caddd" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 36761 } } peers { permanent_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 34509 } }
I20260812 19:02:51.698488 19084 raft_consensus.cc:374] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:51.698549 19084 raft_consensus.cc:729] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e74453211d64703babfc9fc1c7caddd, State: Initialized, Role: FOLLOWER
I20260812 19:02:51.698978 19084 consensus_queue.cc:260] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d0a4820be364949831351cd4a50bba1" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41747 } } peers { permanent_uuid: "2e74453211d64703babfc9fc1c7caddd" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 36761 } } peers { permanent_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 34509 } }
I20260812 19:02:51.699661 19085 raft_consensus.cc:348] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d0a4820be364949831351cd4a50bba1" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41747 } } peers { permanent_uuid: "2e74453211d64703babfc9fc1c7caddd" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 36761 } } peers { permanent_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 34509 } }
I20260812 19:02:51.699911 19085 raft_consensus.cc:374] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:51.699921 19084 ts_tablet_manager.cc:1428] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 19:02:51.699977 19085 raft_consensus.cc:729] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d0a4820be364949831351cd4a50bba1, State: Initialized, Role: FOLLOWER
I20260812 19:02:51.700234 18744 heartbeater.cc:503] Master 127.29.112.254:41141 was elected leader, sending a full tablet report...
I20260812 19:02:51.700398 19085 consensus_queue.cc:260] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d0a4820be364949831351cd4a50bba1" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41747 } } peers { permanent_uuid: "2e74453211d64703babfc9fc1c7caddd" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 36761 } } peers { permanent_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 34509 } }
I20260812 19:02:51.701316 19061 heartbeater.cc:503] Master 127.29.112.254:41141 was elected leader, sending a full tablet report...
I20260812 19:02:51.701442 19085 ts_tablet_manager.cc:1428] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 19:02:51.739351 19091 raft_consensus.cc:482] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:51.739566 19091 raft_consensus.cc:504] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d0a4820be364949831351cd4a50bba1" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41747 } } peers { permanent_uuid: "2e74453211d64703babfc9fc1c7caddd" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 36761 } } peers { permanent_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 34509 } }
I20260812 19:02:51.740629 19091 leader_election.cc:288] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d0a4820be364949831351cd4a50bba1 (127.29.112.195:41747), 36e09c43098e4b43b7a7ce8b37a91bd2 (127.29.112.194:34509)
I20260812 19:02:51.745654 18746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.740859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:41747 (local address 127.29.112.193:45243)
0812 19:02:51.740995 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:51.741011 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:51.741122 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:51.741464 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:51.741466 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:51.741475 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:51.741605 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.741612 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.742701 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.742704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.744351 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.744363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.745210 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.745213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.745292 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.745313 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:02:51.745405 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 19:02:51.745502 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":219}
I20260812 19:02:51.745663 19076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.741053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:45243 (local address 127.29.112.195:41747)
0812 19:02:51.741217 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:51.741222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:51.741240 (+ 18us) server_negotiation.cc:407] Connection header received
0812 19:02:51.741284 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:51.741286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:51.741353 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:51.741448 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:51.741723 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.742558 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.744508 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.745082 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.745206 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.745254 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:51.745448 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 19:02:51.745527 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 19:02:51.745796 19079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.740957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:53349 (local address 127.29.112.194:34509)
0812 19:02:51.741075 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:51.741079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:51.741341 (+ 262us) server_negotiation.cc:407] Connection header received
0812 19:02:51.741430 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:51.741436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:51.741486 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:51.741562 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:51.741859 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.742579 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.744497 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.745082 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.745204 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.745254 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:51.745609 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 19:02:51.745684 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:02:51.746246 19093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.740859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:34509 (local address 127.29.112.193:53349)
0812 19:02:51.741239 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:51.741258 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:51.741340 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:51.741586 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:51.741589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:51.741602 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:51.741718 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.741724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.742701 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.742704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.744351 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:51.744363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.745339 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.745342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:51.745462 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.745485 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 19:02:51.746007 (+ 522us) client_negotiation.cc:240] Negotiation successful
0812 19:02:51.746107 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":314,"thread_start_us":99,"threads_started":1}
I20260812 19:02:51.746402 19008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59a36d1849974a6b8cf62a60bbf49ce3" candidate_uuid: "2e74453211d64703babfc9fc1c7caddd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d0a4820be364949831351cd4a50bba1" is_pre_election: true
I20260812 19:02:51.746774 19008 raft_consensus.cc:2454] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e74453211d64703babfc9fc1c7caddd in term 0.
I20260812 19:02:51.747040 18850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59a36d1849974a6b8cf62a60bbf49ce3" candidate_uuid: "2e74453211d64703babfc9fc1c7caddd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" is_pre_election: true
I20260812 19:02:51.747282 18607 leader_election.cc:302] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e74453211d64703babfc9fc1c7caddd, 7d0a4820be364949831351cd4a50bba1; no voters:
I20260812 19:02:51.747313 18850 raft_consensus.cc:2454] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e74453211d64703babfc9fc1c7caddd in term 0.
I20260812 19:02:51.747573 19091 raft_consensus.cc:2781] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:02:51.747673 19091 raft_consensus.cc:482] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:02:51.747730 19091 raft_consensus.cc:3037] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:51.748764 19091 raft_consensus.cc:504] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d0a4820be364949831351cd4a50bba1" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41747 } } peers { permanent_uuid: "2e74453211d64703babfc9fc1c7caddd" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 36761 } } peers { permanent_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 34509 } }
I20260812 19:02:51.749085 19091 leader_election.cc:288] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [CANDIDATE]: Term 1 election: Requested vote from peers 7d0a4820be364949831351cd4a50bba1 (127.29.112.195:41747), 36e09c43098e4b43b7a7ce8b37a91bd2 (127.29.112.194:34509)
I20260812 19:02:51.749369 19008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59a36d1849974a6b8cf62a60bbf49ce3" candidate_uuid: "2e74453211d64703babfc9fc1c7caddd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d0a4820be364949831351cd4a50bba1"
I20260812 19:02:51.749396 18850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59a36d1849974a6b8cf62a60bbf49ce3" candidate_uuid: "2e74453211d64703babfc9fc1c7caddd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2"
I20260812 19:02:51.749491 19008 raft_consensus.cc:3037] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:51.749528 18850 raft_consensus.cc:3037] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:51.750662 19008 raft_consensus.cc:2454] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e74453211d64703babfc9fc1c7caddd in term 1.
I20260812 19:02:51.750720 18850 raft_consensus.cc:2454] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e74453211d64703babfc9fc1c7caddd in term 1.
I20260812 19:02:51.750988 18607 leader_election.cc:302] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e74453211d64703babfc9fc1c7caddd, 7d0a4820be364949831351cd4a50bba1; no voters:
I20260812 19:02:51.751171 19091 raft_consensus.cc:2781] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:51.751400 19091 raft_consensus.cc:686] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 1 LEADER]: Becoming Leader. State: Replica: 2e74453211d64703babfc9fc1c7caddd, State: Running, Role: LEADER
I20260812 19:02:51.751741 19091 consensus_queue.cc:237] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d0a4820be364949831351cd4a50bba1" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41747 } } peers { permanent_uuid: "2e74453211d64703babfc9fc1c7caddd" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 36761 } } peers { permanent_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 34509 } }
W20260812 19:02:51.752058 18745 tablet.cc:2369] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:02:51.754943 18514 catalog_manager.cc:5495] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e74453211d64703babfc9fc1c7caddd (127.29.112.193). New cstate: current_term: 1 leader_uuid: "2e74453211d64703babfc9fc1c7caddd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d0a4820be364949831351cd4a50bba1" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e74453211d64703babfc9fc1c7caddd" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 36761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 34509 } health_report { overall_health: UNKNOWN } } }
I20260812 19:02:51.773211 19081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.769890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36740 (local address 127.29.112.193:36761)
0812 19:02:51.770038 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:51.770044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:51.770058 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:02:51.770123 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:51.770125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:51.770174 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:51.770274 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:02:51.770507 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.770976 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.771840 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:51.772059 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:51.772167 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:51.772467 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:02:51.772842 (+ 375us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:02:51.772983 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:51.773018 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 19:02:51.773067 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 19:02:51.778626 19008 raft_consensus.cc:1261] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1 [term 1 FOLLOWER]: Refusing update from remote peer 2e74453211d64703babfc9fc1c7caddd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:51.778625 18850 raft_consensus.cc:1261] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2 [term 1 FOLLOWER]: Refusing update from remote peer 2e74453211d64703babfc9fc1c7caddd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:51.779292 19095 consensus_queue.cc:1035] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "36e09c43098e4b43b7a7ce8b37a91bd2" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 34509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:51.779436 19091 consensus_queue.cc:1035] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d0a4820be364949831351cd4a50bba1" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:51.788112 19099 mvcc.cc:205] Tried to move back new op lower bound from 7317755378798268416 to 7317755378698063872. Current Snapshot: MvccSnapshot[applied={T|T < 7317755378798268416}]
I20260812 19:02:51.795361 18515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:51.795536 18515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:51.797953 18515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 253577fb9a3a49558880b032c69ea7e7: Sending DeleteTablet for 3 replicas of tablet 59a36d1849974a6b8cf62a60bbf49ce3
I20260812 19:02:51.798761 18827 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a36d1849974a6b8cf62a60bbf49ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:51 UTC) from {username='slave'} at 127.0.0.1:53484
I20260812 19:02:51.798763 18666 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a36d1849974a6b8cf62a60bbf49ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:51 UTC) from {username='slave'} at 127.0.0.1:36730
I20260812 19:02:51.799218 19114 tablet_replica.cc:330] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd: stopping tablet replica
I20260812 19:02:51.799562 19114 raft_consensus.cc:2229] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:02:51.799558 18986 tablet_service.cc:1514] Processing DeleteTablet for tablet 59a36d1849974a6b8cf62a60bbf49ce3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:51 UTC) from {username='slave'} at 127.0.0.1:44960
I20260812 19:02:51.799891 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 18591
I20260812 19:02:51.799995 19114 raft_consensus.cc:2258] T 59a36d1849974a6b8cf62a60bbf49ce3 P 2e74453211d64703babfc9fc1c7caddd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:51.803607 19113 tablet_replica.cc:330] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2: stopping tablet replica
I20260812 19:02:51.803867 19113 raft_consensus.cc:2229] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:02:51.804152 19113 raft_consensus.cc:2258] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:51.804978 19115 tablet_replica.cc:330] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1: stopping tablet replica
I20260812 19:02:51.805244 19115 raft_consensus.cc:2229] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:02:51.805531 19115 raft_consensus.cc:2258] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:51.805873 19113 ts_tablet_manager.cc:1905] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:51.806782 19115 ts_tablet_manager.cc:1905] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:51.808836 19115 ts_tablet_manager.cc:1918] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:51.808954 19115 log.cc:1196] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/59a36d1849974a6b8cf62a60bbf49ce3
I20260812 19:02:51.809242 19115 ts_tablet_manager.cc:1939] T 59a36d1849974a6b8cf62a60bbf49ce3 P 7d0a4820be364949831351cd4a50bba1: Deleting consensus metadata
I20260812 19:02:51.809508 19113 ts_tablet_manager.cc:1918] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:51.809612 19113 log.cc:1196] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/59a36d1849974a6b8cf62a60bbf49ce3
I20260812 19:02:51.810011 19113 ts_tablet_manager.cc:1939] T 59a36d1849974a6b8cf62a60bbf49ce3 P 36e09c43098e4b43b7a7ce8b37a91bd2: Deleting consensus metadata
I20260812 19:02:51.810429 18495 catalog_manager.cc:4841] TS 7d0a4820be364949831351cd4a50bba1 (127.29.112.195:41747): tablet 59a36d1849974a6b8cf62a60bbf49ce3 (table test-table [id=7eaebd57954447beb40877b115a4fa6a]) successfully deleted
I20260812 19:02:51.811178 18499 catalog_manager.cc:4841] TS 36e09c43098e4b43b7a7ce8b37a91bd2 (127.29.112.194:34509): tablet 59a36d1849974a6b8cf62a60bbf49ce3 (table test-table [id=7eaebd57954447beb40877b115a4fa6a]) successfully deleted
W20260812 19:02:51.811735 18495 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:36761 (error 108)
I20260812 19:02:51.812433 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 18747
I20260812 19:02:51.812734 19078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:51.812395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:36761 (local address 127.0.0.1:36746)
0812 19:02:51.812561 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:51.812639 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 19:02:51.818672 18495 catalog_manager.cc:4584] TS 2e74453211d64703babfc9fc1c7caddd (127.29.112.193:36761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 59a36d1849974a6b8cf62a60bbf49ce3: Network error: Client connection negotiation failed: client connection to 127.29.112.193:36761: connect: Connection refused (error 111)
I20260812 19:02:51.823498 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 18905
I20260812 19:02:51.831637 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 18476
2026-08-12T19:02:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1031 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:02:51 dist-test-slave-6nmv krb5kdc[19120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:02:51 dist-test-slave-6nmv krb5kdc[19120](info): set up 2 sockets
Aug 12 19:02:51 dist-test-slave-6nmv krb5kdc[19120](info): commencing operation
krb5kdc: starting...
W20260812 19:02:52.922438 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:02:53 dist-test-slave-6nmv krb5kdc[19120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:02:53.040802 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:41419
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:41419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:53.144665 19136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:53.144869 19136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:53.144908 19136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:53.148785 19136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:02:53.148828 19136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:53.148844 19136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:53.148859 19136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:02:53.148872 19136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:02:53.153656 19136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:41419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:41419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:53.154539 19136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:53.155403 19136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:53.162933 19142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:53.163107 19136 server_base.cc:1034] running on GCE node
W20260812 19:02:53.163034 19144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:53.162932 19146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:53.163720 19136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:53.164675 19136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:53.165869 19136 hybrid_clock.cc:648] HybridClock initialized: now 1786561373165849 us; error 34 us; skew 500 ppm
Aug 12 19:02:53 dist-test-slave-6nmv krb5kdc[19120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561373, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:53.168401 19136 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:02:53.169155 19136 webserver.cc:460] Webserver started at http://127.29.112.254:44935/ using document root <none> and password file <none>
I20260812 19:02:53.169706 19136 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:53.169787 19136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:53.170020 19136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:53.171777 19136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "9ae6f8be38a84f999887b7f333e5212e"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "5a3ec0db5ee78e65e65ddb3a07d09cbf"
server_key_iv: "6f22a574dcb99f9f3e904107ef9af86a"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.172356 19136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "9ae6f8be38a84f999887b7f333e5212e"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "5a3ec0db5ee78e65e65ddb3a07d09cbf"
server_key_iv: "6f22a574dcb99f9f3e904107ef9af86a"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.175769 19136 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 19:02:53.178007 19156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:53.179075 19136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:02:53.179198 19136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "9ae6f8be38a84f999887b7f333e5212e"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "5a3ec0db5ee78e65e65ddb3a07d09cbf"
server_key_iv: "6f22a574dcb99f9f3e904107ef9af86a"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.179311 19136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:53.202854 19136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:53.205334 19136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:53.205523 19136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:53.212109 19136 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:41419
I20260812 19:02:53.212157 19219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:41419 every 8 connection(s)
I20260812 19:02:53.212975 19136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:02:53.215500 19220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:53.216042 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 19136
I20260812 19:02:53.216183 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:02:53.216485 30147 external_mini_cluster.cc:1428] Setting key 7014eaf174cda44fcc77f1102dfab695
I20260812 19:02:53.221493 19220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e: Bootstrap starting.
I20260812 19:02:53.224026 19220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:53.224781 19220 log.cc:824] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:02:53 dist-test-slave-6nmv krb5kdc[19120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:53.226770 19220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e: No bootstrap required, opened a new log
I20260812 19:02:53.229708 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.217723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.29.112.254:41419)
0812 19:02:53.218200 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:53.218213 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:53.218246 (+ 33us) server_negotiation.cc:407] Connection header received
0812 19:02:53.218846 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:53.218861 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:53.219184 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:53.219451 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:53.220375 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.221215 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.222323 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.222774 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.222882 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.225467 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:53.225472 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:53.225484 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:53.225510 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:53.227180 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:53.227605 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:53.227610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:53.227614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:53.227720 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:53.228010 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:53.228014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:53.228016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:53.228364 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:53.228552 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:53.228813 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 19:02:53.229073 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":112,"threads_started":1}
I20260812 19:02:53.230122 19220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ae6f8be38a84f999887b7f333e5212e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41419 } }
I20260812 19:02:53.230310 19220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:53.230374 19220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae6f8be38a84f999887b7f333e5212e, State: Initialized, Role: FOLLOWER
I20260812 19:02:53.230820 19220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ae6f8be38a84f999887b7f333e5212e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41419 } }
I20260812 19:02:53.230978 19220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:02:53.231041 19220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:02:53.231135 19220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:53.232177 19220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ae6f8be38a84f999887b7f333e5212e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41419 } }
I20260812 19:02:53.232446 19220 leader_election.cc:302] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ae6f8be38a84f999887b7f333e5212e; no voters:
I20260812 19:02:53.232674 19220 leader_election.cc:288] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:02:53.232798 19226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:53.233031 19226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [term 1 LEADER]: Becoming Leader. State: Replica: 9ae6f8be38a84f999887b7f333e5212e, State: Running, Role: LEADER
I20260812 19:02:53.233348 19226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ae6f8be38a84f999887b7f333e5212e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41419 } }
I20260812 19:02:53.233798 19220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:02:53.235064 19229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ae6f8be38a84f999887b7f333e5212e. Latest consensus state: current_term: 1 leader_uuid: "9ae6f8be38a84f999887b7f333e5212e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ae6f8be38a84f999887b7f333e5212e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41419 } } }
I20260812 19:02:53.235179 19229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:53.235550 19237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:02:53.236095 19228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ae6f8be38a84f999887b7f333e5212e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ae6f8be38a84f999887b7f333e5212e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 41419 } } }
I20260812 19:02:53.236196 19228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:53.238770 19237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:02:53.244139 19237 catalog_manager.cc:1347] Generated new cluster ID: ab2474c597b54d0398033c9faf55182a
I20260812 19:02:53.244205 19237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:02:53.263883 19237 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:02:53.265151 19237 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:02:53.271036 19237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e: Generated new TSK 0
I20260812 19:02:53.271767 19237 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 19:02:53.302294 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41419
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:53.407176 19251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:53.407407 19251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:53.407449 19251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:53.411284 19251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:53.411332 19251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:53.411417 19251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:02:53.416457 19251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:53.417455 19251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:53.418419 19251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:53.427186 19263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:53.427224 19259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:53.427258 19251 server_base.cc:1034] running on GCE node
W20260812 19:02:53.427464 19260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:53.427820 19251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:53.428391 19251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:53.429581 19251 hybrid_clock.cc:648] HybridClock initialized: now 1786561373429569 us; error 28 us; skew 500 ppm
Aug 12 19:02:53 dist-test-slave-6nmv krb5kdc[19120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561373, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:53.432252 19251 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:02:53.433072 19251 webserver.cc:460] Webserver started at http://127.29.112.193:42965/ using document root <none> and password file <none>
I20260812 19:02:53.433527 19251 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:53.433581 19251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:53.433724 19251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:53.435643 19251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "22449ae8d44b4d519d535157b5dcb8e0"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "3e7247e8b04f7707790d9c0d2acd2c28"
server_key_iv: "a874e000e4b2eb850b04b415ed77de47"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.436120 19251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "22449ae8d44b4d519d535157b5dcb8e0"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "3e7247e8b04f7707790d9c0d2acd2c28"
server_key_iv: "a874e000e4b2eb850b04b415ed77de47"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.439585 19251 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 19:02:53.441619 19270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:53.442746 19251 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:02:53.442854 19251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "22449ae8d44b4d519d535157b5dcb8e0"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "3e7247e8b04f7707790d9c0d2acd2c28"
server_key_iv: "a874e000e4b2eb850b04b415ed77de47"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.442942 19251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:53.462296 19251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:53.464748 19251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:53.464936 19251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:53.465443 19251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:53.466168 19251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:53.466210 19251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:53.466267 19251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:53.466296 19251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:53.475397 19251 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:45507
I20260812 19:02:53.475427 19410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:45507 every 8 connection(s)
I20260812 19:02:53.476166 19251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:02:53.477967 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 19251
I20260812 19:02:53.478101 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 19:02:53.478428 30147 external_mini_cluster.cc:1428] Setting key 14586dc29a655d2d5327b62700e70602
Aug 12 19:02:53 dist-test-slave-6nmv krb5kdc[19120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561373, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 19:02:53.490391 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.477776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:38775 (local address 127.29.112.254:41419)
0812 19:02:53.477927 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:53.477930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:53.478927 (+ 997us) server_negotiation.cc:407] Connection header received
0812 19:02:53.480474 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:53.480477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:53.480528 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:53.480602 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:53.481662 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.482300 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.483539 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.483767 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.483867 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.486705 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:53.486709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:53.486711 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:53.486733 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:53.488168 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:53.488756 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:53.488758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:53.488760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:53.488835 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:53.489179 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:53.489182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:53.489183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:53.489504 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:53.489603 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:53.490088 (+ 485us) server_negotiation.cc:299] Negotiation successful
0812 19:02:53.490201 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
I20260812 19:02:53.491320 19413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.478259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:41419 (local address 127.29.112.193:38775)
0812 19:02:53.478756 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:53.478812 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:53.480286 (+ 1474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:53.480752 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:53.480758 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:53.481087 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:53.481513 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.481526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.482424 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.482429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.483425 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.483434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.483922 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.483926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.484083 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.484669 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:53.484684 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:53.486515 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:53.488288 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:53.488296 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:53.488314 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:53.488662 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:53.488907 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:53.488909 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:53.488910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:53.489041 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:53.489603 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:53.489607 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:53.489868 (+ 261us) client_negotiation.cc:769] Sending connection context
0812 19:02:53.490026 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 19:02:53.490328 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":139,"threads_started":1}
I20260812 19:02:53.492494 19411 heartbeater.cc:347] Connected to a master server at 127.29.112.254:41419
I20260812 19:02:53.492748 19411 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:53.493320 19411 heartbeater.cc:511] Master 127.29.112.254:41419 requested a full tablet report, sending...
I20260812 19:02:53.494837 19173 ts_manager.cc:194] Registered new tserver with Master: 22449ae8d44b4d519d535157b5dcb8e0 (127.29.112.193:45507)
I20260812 19:02:53.495882 19173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:38775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 19:02:53.506399 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41419
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:53.612495 19417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:53.612761 19417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:53.612845 19417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:53.616549 19417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:53.616621 19417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:53.616721 19417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:02:53.621300 19417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:53.622193 19417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:53.623302 19417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:53.632484 19427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:53.632584 19425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:53.632469 19423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:53.632988 19417 server_base.cc:1034] running on GCE node
I20260812 19:02:53.633348 19417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:53.633975 19417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:53.635147 19417 hybrid_clock.cc:648] HybridClock initialized: now 1786561373635134 us; error 25 us; skew 500 ppm
Aug 12 19:02:53 dist-test-slave-6nmv krb5kdc[19120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561373, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:53.637943 19417 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:02:53.638664 19417 webserver.cc:460] Webserver started at http://127.29.112.194:42875/ using document root <none> and password file <none>
I20260812 19:02:53.639137 19417 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:53.639189 19417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:53.639325 19417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:53.641108 19417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "06b95f595131d99762cd1a76275cc55e"
server_key_iv: "276592b5209ee7c14e6357695c31264b"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.641556 19417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "06b95f595131d99762cd1a76275cc55e"
server_key_iv: "276592b5209ee7c14e6357695c31264b"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.644735 19417 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 19:02:53.646996 19437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:53.648125 19417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:02:53.648255 19417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "06b95f595131d99762cd1a76275cc55e"
server_key_iv: "276592b5209ee7c14e6357695c31264b"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.648355 19417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:53.662561 19417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:53.665151 19417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:53.665343 19417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:53.665889 19417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:53.666651 19417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:53.666719 19417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:53.666786 19417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:53.666826 19417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:53.675381 19417 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:45869
I20260812 19:02:53.675422 19572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:45869 every 8 connection(s)
I20260812 19:02:53.676138 19417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:02:53.682210 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 19417
I20260812 19:02:53.682363 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 19:02:53.682690 30147 external_mini_cluster.cc:1428] Setting key 2c9375737b1bf3bd48e7305c0d76ef74
Aug 12 19:02:53 dist-test-slave-6nmv krb5kdc[19120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561373, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:53.690739 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.677788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:45935 (local address 127.29.112.254:41419)
0812 19:02:53.677938 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:53.677949 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:53.678755 (+ 806us) server_negotiation.cc:407] Connection header received
0812 19:02:53.680118 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:53.680122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:53.680179 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:53.680303 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:53.681330 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.681988 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.683258 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.683582 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.683671 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.686443 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:53.686447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:53.686450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:53.686484 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:53.688009 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:53.688585 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:53.688591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:53.688593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:53.688688 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:53.689064 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:53.689070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:53.689072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:53.689296 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:53.689431 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:53.690306 (+ 875us) server_negotiation.cc:299] Negotiation successful
0812 19:02:53.690523 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 19:02:53.691506 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.678163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:41419 (local address 127.29.112.194:45935)
0812 19:02:53.678599 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:53.678650 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:53.679901 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:53.680457 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:53.680464 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:53.680769 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:53.681184 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.681196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.682135 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.682138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.683139 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.683151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.683741 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.683745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.683928 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.684536 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:53.684552 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:53.686251 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:53.688153 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:53.688160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:53.688174 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:53.688463 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:53.688798 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:53.688801 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:53.688803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:53.688959 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:53.689431 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:53.689439 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:53.689776 (+ 337us) client_negotiation.cc:769] Sending connection context
0812 19:02:53.690231 (+ 455us) client_negotiation.cc:240] Negotiation successful
0812 19:02:53.690584 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":114,"threads_started":1}
I20260812 19:02:53.692728 19573 heartbeater.cc:347] Connected to a master server at 127.29.112.254:41419
I20260812 19:02:53.693017 19573 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
I20260812 19:02:53.693634 19573 heartbeater.cc:511] Master 127.29.112.254:41419 requested a full tablet report, sending...
I20260812 19:02:53.694751 19173 ts_manager.cc:194] Registered new tserver with Master: dc57cdc9b9ee4bfea727b6a2d7f6dd3c (127.29.112.194:45869)
I20260812 19:02:53.695304 19173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:45935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 19:02:53.712934 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41419
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:53.821173 19578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:53.821393 19578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:53.821431 19578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:53.825403 19578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:53.825460 19578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:53.825515 19578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:02:53.830731 19578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:41419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:53.831645 19578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:53.832561 19578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:53.841889 19583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:53.841858 19588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:53.842131 19578 server_base.cc:1034] running on GCE node
W20260812 19:02:53.842336 19584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:53.842748 19578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:53.843398 19578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:53.844594 19578 hybrid_clock.cc:648] HybridClock initialized: now 1786561373844580 us; error 25 us; skew 500 ppm
Aug 12 19:02:53 dist-test-slave-6nmv krb5kdc[19120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561373, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:53.847503 19578 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:02:53.848346 19578 webserver.cc:460] Webserver started at http://127.29.112.195:41277/ using document root <none> and password file <none>
I20260812 19:02:53.848908 19578 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:53.848997 19578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:53.849212 19578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:53.851289 19578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "de37a6b64b1d43779d2a632ed8c4042d"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "54365e73f2e8f4c01daf86cfaad1b5ea"
server_key_iv: "13d4e99eea3ea343ac1a3fc7f46f3aea"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.851905 19578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "de37a6b64b1d43779d2a632ed8c4042d"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "54365e73f2e8f4c01daf86cfaad1b5ea"
server_key_iv: "13d4e99eea3ea343ac1a3fc7f46f3aea"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.855654 19578 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 19:02:53.858181 19598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:53.859411 19578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 19:02:53.859536 19578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "de37a6b64b1d43779d2a632ed8c4042d"
format_stamp: "Formatted at 2026-08-12 19:02:53 on dist-test-slave-6nmv"
server_key: "54365e73f2e8f4c01daf86cfaad1b5ea"
server_key_iv: "13d4e99eea3ea343ac1a3fc7f46f3aea"
server_key_version: "encryptionkey@0"
I20260812 19:02:53.859640 19578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:53.883620 19578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:53.888255 19578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:53.888422 19578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:53.888921 19578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:53.889740 19578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:53.889787 19578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:53.889878 19578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:53.889910 19578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:53.898357 19578 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:41867
I20260812 19:02:53.898401 19741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:41867 every 8 connection(s)
I20260812 19:02:53.899262 19578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
Aug 12 19:02:53 dist-test-slave-6nmv krb5kdc[19120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561373, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:53.909598 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 19578
I20260812 19:02:53.909740 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 19:02:53.910051 30147 external_mini_cluster.cc:1428] Setting key 7e1c7459d8c2deea3785ace580fb9fc0
I20260812 19:02:53.913035 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.901069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:52305 (local address 127.29.112.254:41419)
0812 19:02:53.901225 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:53.901229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:53.902073 (+ 844us) server_negotiation.cc:407] Connection header received
0812 19:02:53.903434 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:53.903436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:53.903482 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:53.903545 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:53.904673 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.905228 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.906416 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.906624 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.906720 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.909256 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:53.909259 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:53.909261 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:53.909307 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:53.910992 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:53.911539 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:53.911544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:53.911546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:53.911643 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:53.911995 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:53.912000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:53.912002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:53.912182 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:53.912285 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:53.912738 (+ 453us) server_negotiation.cc:299] Negotiation successful
0812 19:02:53.912856 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 19:02:53.913751 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.901416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:41419 (local address 127.29.112.195:52305)
0812 19:02:53.901933 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:53.901976 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:53.903226 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:53.903697 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:53.903703 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:53.904057 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:53.904517 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.904529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.905338 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.905340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.906302 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.906312 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.906769 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.906772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.906939 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.907488 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:53.907505 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:53.909082 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:53.911113 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:53.911119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:53.911132 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:53.911426 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:53.911747 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:53.911750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:53.911752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:53.911890 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:53.912283 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:53.912289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:53.912520 (+ 231us) client_negotiation.cc:769] Sending connection context
0812 19:02:53.912695 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 19:02:53.912997 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260812 19:02:53.914955 19742 heartbeater.cc:347] Connected to a master server at 127.29.112.254:41419
I20260812 19:02:53.915174 19742 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:53.915663 19742 heartbeater.cc:511] Master 127.29.112.254:41419 requested a full tablet report, sending...
I20260812 19:02:53.916613 19173 ts_manager.cc:194] Registered new tserver with Master: de37a6b64b1d43779d2a632ed8c4042d (127.29.112.195:41867)
I20260812 19:02:53.917182 19173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:52305
I20260812 19:02:53.923859 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:02:53.934202 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.925967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.29.112.254:41419)
0812 19:02:53.926123 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:53.926129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:53.926488 (+ 359us) server_negotiation.cc:407] Connection header received
0812 19:02:53.926614 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:53.926617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:53.926663 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:53.926737 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:53.927492 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.927992 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.928765 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.928976 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.929056 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.930657 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:53.930660 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:53.930663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:53.930687 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:53.932277 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:53.932748 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:53.932751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:53.932752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:53.932837 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:53.933210 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:53.933212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:53.933213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:53.933522 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:53.933643 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:53.933936 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 19:02:53.934034 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 19:02:53.937067 19173 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:02:53.940239 19173 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:02:53.955469 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.949163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45140 (local address 127.29.112.193:45507)
0812 19:02:53.949539 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:53.949555 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:53.949578 (+ 23us) server_negotiation.cc:407] Connection header received
0812 19:02:53.949670 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:53.949677 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:53.949964 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:53.950120 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:53.950694 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.951920 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.953634 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.954588 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.954709 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.954845 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:53.955206 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 19:02:53.955308 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":88,"threads_started":1}
I20260812 19:02:53.955459 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.948446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45507 (local address 127.0.0.1:45140)
0812 19:02:53.948769 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:53.948819 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:53.948954 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:53.950295 (+ 1341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:53.950301 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:53.950326 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:53.950556 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.950564 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.952134 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.952139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.953477 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.953486 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.954941 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.954945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.955034 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.955062 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 19:02:53.955173 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 19:02:53.955300 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":77,"threads_started":1}
I20260812 19:02:53.955953 19757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.948780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:45869 (local address 127.0.0.1:48816)
0812 19:02:53.950146 (+ 1366us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:53.950156 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:53.950261 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:53.950899 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:53.950902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:53.950919 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:53.951038 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.951045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.952971 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.952974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.954311 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.954321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.955569 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.955573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.955662 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.955683 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:02:53.955747 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 19:02:53.955808 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1317,"mutex_wait_us":67,"thread_start_us":89,"threads_started":1}
I20260812 19:02:53.956395 19762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.949220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60934 (local address 127.29.112.195:41867)
0812 19:02:53.950585 (+ 1365us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:53.950597 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:53.950653 (+ 56us) server_negotiation.cc:407] Connection header received
0812 19:02:53.950754 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:53.950760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:53.951024 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:53.951502 (+ 478us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:53.951685 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.952838 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.954530 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.955265 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.955390 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.955505 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:53.956028 (+ 523us) server_negotiation.cc:299] Negotiation successful
0812 19:02:53.956249 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":134,"threads_started":1}
I20260812 19:02:53.956650 19761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.949191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48816 (local address 127.29.112.194:45869)
0812 19:02:53.950011 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:53.950025 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:53.950242 (+ 217us) server_negotiation.cc:407] Connection header received
0812 19:02:53.950370 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:53.950375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:53.950557 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:53.950708 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:53.951185 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.952825 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.954444 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.955249 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.955390 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.955505 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:53.956215 (+ 710us) server_negotiation.cc:299] Negotiation successful
0812 19:02:53.956317 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":140,"threads_started":1}
I20260812 19:02:53.956139 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:53.948446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:41867 (local address 127.0.0.1:60934)
0812 19:02:53.949569 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:53.949579 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:53.949663 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:53.951263 (+ 1600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:53.951265 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:53.951285 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:53.951363 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.951371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.952971 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.952974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.954397 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:53.954406 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:53.955775 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:53.955779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:53.955852 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:53.955871 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 19:02:53.955945 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 19:02:53.956022 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"mutex_wait_us":272,"thread_start_us":87,"threads_started":1}
I20260812 19:02:53.958583 19328 tablet_service.cc:1467] Processing CreateTablet for tablet 73395b703d954a17bb13b137fcbc75e6 (DEFAULT_TABLE table=test-table [id=e2862878ca8242f7917238bc60c157c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:53.959503 19328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73395b703d954a17bb13b137fcbc75e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:53.959389 19491 tablet_service.cc:1467] Processing CreateTablet for tablet 73395b703d954a17bb13b137fcbc75e6 (DEFAULT_TABLE table=test-table [id=e2862878ca8242f7917238bc60c157c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:53.959342 19656 tablet_service.cc:1467] Processing CreateTablet for tablet 73395b703d954a17bb13b137fcbc75e6 (DEFAULT_TABLE table=test-table [id=e2862878ca8242f7917238bc60c157c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:53.960253 19491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73395b703d954a17bb13b137fcbc75e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:53.960253 19656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73395b703d954a17bb13b137fcbc75e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:53.965963 19763 tablet_bootstrap.cc:492] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0: Bootstrap starting.
I20260812 19:02:53.967393 19764 tablet_bootstrap.cc:492] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c: Bootstrap starting.
I20260812 19:02:53.967393 19765 tablet_bootstrap.cc:492] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d: Bootstrap starting.
I20260812 19:02:53.968094 19763 tablet_bootstrap.cc:654] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:53.969128 19763 log.cc:824] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:53.969404 19764 tablet_bootstrap.cc:654] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:53.969404 19765 tablet_bootstrap.cc:654] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:53.970647 19765 log.cc:824] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:53.970647 19764 log.cc:824] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:53.970993 19763 tablet_bootstrap.cc:492] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0: No bootstrap required, opened a new log
I20260812 19:02:53.971206 19763 ts_tablet_manager.cc:1397] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:02:53.972312 19764 tablet_bootstrap.cc:492] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c: No bootstrap required, opened a new log
I20260812 19:02:53.972517 19764 ts_tablet_manager.cc:1397] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:02:53.972605 19765 tablet_bootstrap.cc:492] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d: No bootstrap required, opened a new log
I20260812 19:02:53.972777 19765 ts_tablet_manager.cc:1397] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:02:53.975032 19763 raft_consensus.cc:348] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de37a6b64b1d43779d2a632ed8c4042d" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41867 } } peers { permanent_uuid: "22449ae8d44b4d519d535157b5dcb8e0" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45507 } } peers { permanent_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45869 } }
I20260812 19:02:53.975287 19763 raft_consensus.cc:374] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:53.975381 19763 raft_consensus.cc:729] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22449ae8d44b4d519d535157b5dcb8e0, State: Initialized, Role: FOLLOWER
I20260812 19:02:53.975217 19764 raft_consensus.cc:348] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de37a6b64b1d43779d2a632ed8c4042d" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41867 } } peers { permanent_uuid: "22449ae8d44b4d519d535157b5dcb8e0" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45507 } } peers { permanent_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45869 } }
I20260812 19:02:53.975468 19764 raft_consensus.cc:374] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:53.975589 19764 raft_consensus.cc:729] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc57cdc9b9ee4bfea727b6a2d7f6dd3c, State: Initialized, Role: FOLLOWER
I20260812 19:02:53.975829 19763 consensus_queue.cc:260] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de37a6b64b1d43779d2a632ed8c4042d" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41867 } } peers { permanent_uuid: "22449ae8d44b4d519d535157b5dcb8e0" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45507 } } peers { permanent_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45869 } }
I20260812 19:02:53.976034 19764 consensus_queue.cc:260] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de37a6b64b1d43779d2a632ed8c4042d" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41867 } } peers { permanent_uuid: "22449ae8d44b4d519d535157b5dcb8e0" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45507 } } peers { permanent_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45869 } }
I20260812 19:02:53.976349 19765 raft_consensus.cc:348] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de37a6b64b1d43779d2a632ed8c4042d" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41867 } } peers { permanent_uuid: "22449ae8d44b4d519d535157b5dcb8e0" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45507 } } peers { permanent_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45869 } }
I20260812 19:02:53.976519 19411 heartbeater.cc:503] Master 127.29.112.254:41419 was elected leader, sending a full tablet report...
I20260812 19:02:53.976632 19765 raft_consensus.cc:374] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:53.976750 19765 raft_consensus.cc:729] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de37a6b64b1d43779d2a632ed8c4042d, State: Initialized, Role: FOLLOWER
I20260812 19:02:53.976890 19763 ts_tablet_manager.cc:1428] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:02:53.977092 19764 ts_tablet_manager.cc:1428] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260812 19:02:53.977190 19412 tablet.cc:2369] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:02:53.977599 19573 heartbeater.cc:503] Master 127.29.112.254:41419 was elected leader, sending a full tablet report...
I20260812 19:02:53.977950 19765 consensus_queue.cc:260] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de37a6b64b1d43779d2a632ed8c4042d" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41867 } } peers { permanent_uuid: "22449ae8d44b4d519d535157b5dcb8e0" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45507 } } peers { permanent_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45869 } }
I20260812 19:02:53.978564 19742 heartbeater.cc:503] Master 127.29.112.254:41419 was elected leader, sending a full tablet report...
I20260812 19:02:53.979633 19765 ts_tablet_manager.cc:1428] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260812 19:02:54.028304 19773 raft_consensus.cc:482] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:54.028517 19773 raft_consensus.cc:504] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de37a6b64b1d43779d2a632ed8c4042d" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41867 } } peers { permanent_uuid: "22449ae8d44b4d519d535157b5dcb8e0" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45507 } } peers { permanent_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45869 } }
I20260812 19:02:54.029801 19773 leader_election.cc:288] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de37a6b64b1d43779d2a632ed8c4042d (127.29.112.195:41867), 22449ae8d44b4d519d535157b5dcb8e0 (127.29.112.193:45507)
I20260812 19:02:54.033334 19762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:54.030117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:40903 (local address 127.29.112.195:41867)
0812 19:02:54.030252 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:54.030258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:54.030277 (+ 19us) server_negotiation.cc:407] Connection header received
0812 19:02:54.030332 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:54.030335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:54.030393 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:54.030497 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:54.030792 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.031297 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.032387 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.032778 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.032915 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:54.032978 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:54.033156 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 19:02:54.033224 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:02:54.033337 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:54.030001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:41867 (local address 127.29.112.194:40903)
0812 19:02:54.030139 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:54.030156 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:54.030247 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:54.030510 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:54.030513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:54.030532 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:54.030676 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:54.030681 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.031454 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.031457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:54.032253 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:54.032262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.032910 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.032913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:54.033021 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:54.033046 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 19:02:54.033126 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 19:02:54.033202 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":6,"spinlock_wait_cycles":36224}
I20260812 19:02:54.033488 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:54.030292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:36075 (local address 127.29.112.193:45507)
0812 19:02:54.030446 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:54.030451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:54.030542 (+ 91us) server_negotiation.cc:407] Connection header received
0812 19:02:54.030697 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:54.030705 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:54.030748 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:54.030822 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:54.031088 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.031575 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.032588 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.032981 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.033101 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:54.033148 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:54.033312 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 19:02:54.033377 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 19:02:54.033502 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:54.030111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45507 (local address 127.29.112.194:36075)
0812 19:02:54.030458 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:54.030469 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:54.030599 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:54.030841 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:54.030844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:54.030859 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:54.030985 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:54.030990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.031698 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.031702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:54.032469 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:54.032478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.033100 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.033104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:54.033197 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:54.033219 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:02:54.033298 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 19:02:54.033361 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"spinlock_wait_cycles":768,"thread_start_us":128,"threads_started":1}
I20260812 19:02:54.034219 19683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73395b703d954a17bb13b137fcbc75e6" candidate_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de37a6b64b1d43779d2a632ed8c4042d" is_pre_election: true
I20260812 19:02:54.034375 19353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73395b703d954a17bb13b137fcbc75e6" candidate_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22449ae8d44b4d519d535157b5dcb8e0" is_pre_election: true
I20260812 19:02:54.034514 19683 raft_consensus.cc:2454] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc57cdc9b9ee4bfea727b6a2d7f6dd3c in term 0.
I20260812 19:02:54.034744 19353 raft_consensus.cc:2454] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc57cdc9b9ee4bfea727b6a2d7f6dd3c in term 0.
I20260812 19:02:54.035041 19440 leader_election.cc:302] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc57cdc9b9ee4bfea727b6a2d7f6dd3c, de37a6b64b1d43779d2a632ed8c4042d; no voters:
I20260812 19:02:54.035331 19773 raft_consensus.cc:2781] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:02:54.035422 19773 raft_consensus.cc:482] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:02:54.035467 19773 raft_consensus.cc:3037] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:54.036528 19773 raft_consensus.cc:504] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de37a6b64b1d43779d2a632ed8c4042d" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41867 } } peers { permanent_uuid: "22449ae8d44b4d519d535157b5dcb8e0" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45507 } } peers { permanent_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45869 } }
I20260812 19:02:54.036885 19773 leader_election.cc:288] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [CANDIDATE]: Term 1 election: Requested vote from peers de37a6b64b1d43779d2a632ed8c4042d (127.29.112.195:41867), 22449ae8d44b4d519d535157b5dcb8e0 (127.29.112.193:45507)
I20260812 19:02:54.037334 19683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73395b703d954a17bb13b137fcbc75e6" candidate_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de37a6b64b1d43779d2a632ed8c4042d"
I20260812 19:02:54.037475 19683 raft_consensus.cc:3037] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:54.037503 19353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73395b703d954a17bb13b137fcbc75e6" candidate_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22449ae8d44b4d519d535157b5dcb8e0"
I20260812 19:02:54.037629 19353 raft_consensus.cc:3037] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:54.038748 19353 raft_consensus.cc:2454] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc57cdc9b9ee4bfea727b6a2d7f6dd3c in term 1.
I20260812 19:02:54.038884 19683 raft_consensus.cc:2454] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc57cdc9b9ee4bfea727b6a2d7f6dd3c in term 1.
I20260812 19:02:54.039150 19438 leader_election.cc:302] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22449ae8d44b4d519d535157b5dcb8e0, dc57cdc9b9ee4bfea727b6a2d7f6dd3c; no voters:
I20260812 19:02:54.039351 19773 raft_consensus.cc:2781] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:54.039541 19773 raft_consensus.cc:686] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 1 LEADER]: Becoming Leader. State: Replica: dc57cdc9b9ee4bfea727b6a2d7f6dd3c, State: Running, Role: LEADER
I20260812 19:02:54.039870 19773 consensus_queue.cc:237] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de37a6b64b1d43779d2a632ed8c4042d" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41867 } } peers { permanent_uuid: "22449ae8d44b4d519d535157b5dcb8e0" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45507 } } peers { permanent_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45869 } }
I20260812 19:02:54.042924 19171 catalog_manager.cc:5495] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c reported cstate change: term changed from 0 to 1, leader changed from <none> to dc57cdc9b9ee4bfea727b6a2d7f6dd3c (127.29.112.194). New cstate: current_term: 1 leader_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de37a6b64b1d43779d2a632ed8c4042d" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22449ae8d44b4d519d535157b5dcb8e0" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc57cdc9b9ee4bfea727b6a2d7f6dd3c" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 45869 } health_report { overall_health: HEALTHY } } }
I20260812 19:02:54.076552 19761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:54.073213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48822 (local address 127.29.112.194:45869)
0812 19:02:54.073337 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:54.073340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:54.073354 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:02:54.073468 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:54.073470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:54.073521 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:54.073621 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:02:54.073825 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.074330 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.075181 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.075384 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.075454 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:54.075750 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:02:54.076129 (+ 379us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:02:54.076233 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:54.076350 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 19:02:54.076383 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:02:54.082465 19683 raft_consensus.cc:1261] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d [term 1 FOLLOWER]: Refusing update from remote peer dc57cdc9b9ee4bfea727b6a2d7f6dd3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:54.082468 19353 raft_consensus.cc:1261] T 73395b703d954a17bb13b137fcbc75e6 P 22449ae8d44b4d519d535157b5dcb8e0 [term 1 FOLLOWER]: Refusing update from remote peer dc57cdc9b9ee4bfea727b6a2d7f6dd3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:54.083300 19773 consensus_queue.cc:1035] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "de37a6b64b1d43779d2a632ed8c4042d" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 41867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:54.083549 19777 consensus_queue.cc:1035] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "22449ae8d44b4d519d535157b5dcb8e0" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:54.093591 19783 mvcc.cc:205] Tried to move back new op lower bound from 7317755388233981952 to 7317755388070133760. Current Snapshot: MvccSnapshot[applied={T|T < 7317755388233981952}]
I20260812 19:02:54.100417 19173 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:54.100553 19173 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:54.102985 19173 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ae6f8be38a84f999887b7f333e5212e: Sending DeleteTablet for 3 replicas of tablet 73395b703d954a17bb13b137fcbc75e6
I20260812 19:02:54.103739 19656 tablet_service.cc:1514] Processing DeleteTablet for tablet 73395b703d954a17bb13b137fcbc75e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:54 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:60934
I20260812 19:02:54.103875 19491 tablet_service.cc:1514] Processing DeleteTablet for tablet 73395b703d954a17bb13b137fcbc75e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:54 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:48816
I20260812 19:02:54.104162 19793 tablet_replica.cc:330] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d: stopping tablet replica
I20260812 19:02:54.104206 19328 tablet_service.cc:1514] Processing DeleteTablet for tablet 73395b703d954a17bb13b137fcbc75e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:54 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:45140
I20260812 19:02:54.104276 19794 tablet_replica.cc:330] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c: stopping tablet replica
I20260812 19:02:54.104497 19793 raft_consensus.cc:2229] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:02:54.104532 19794 raft_consensus.cc:2229] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:02:54.104797 19793 raft_consensus.cc:2258] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:54.104895 19794 raft_consensus.cc:2258] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:54.105495 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 19251
I20260812 19:02:54.106675 19794 ts_tablet_manager.cc:1905] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:54.106822 19793 ts_tablet_manager.cc:1905] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:54.109440 19794 ts_tablet_manager.cc:1918] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:54.109557 19794 log.cc:1196] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/73395b703d954a17bb13b137fcbc75e6
I20260812 19:02:54.109954 19794 ts_tablet_manager.cc:1939] T 73395b703d954a17bb13b137fcbc75e6 P dc57cdc9b9ee4bfea727b6a2d7f6dd3c: Deleting consensus metadata
I20260812 19:02:54.110311 19793 ts_tablet_manager.cc:1918] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:54.110448 19793 log.cc:1196] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/73395b703d954a17bb13b137fcbc75e6
I20260812 19:02:54.110806 19793 ts_tablet_manager.cc:1939] T 73395b703d954a17bb13b137fcbc75e6 P de37a6b64b1d43779d2a632ed8c4042d: Deleting consensus metadata
I20260812 19:02:54.112006 19159 catalog_manager.cc:4841] TS de37a6b64b1d43779d2a632ed8c4042d (127.29.112.195:41867): tablet 73395b703d954a17bb13b137fcbc75e6 (table test-table [id=e2862878ca8242f7917238bc60c157c3]) successfully deleted
I20260812 19:02:54.112576 19158 catalog_manager.cc:4841] TS dc57cdc9b9ee4bfea727b6a2d7f6dd3c (127.29.112.194:45869): tablet 73395b703d954a17bb13b137fcbc75e6 (table test-table [id=e2862878ca8242f7917238bc60c157c3]) successfully deleted
W20260812 19:02:54.117787 19157 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:45507 (error 108)
I20260812 19:02:54.118623 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 19417
I20260812 19:02:54.118803 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:54.118523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45507 (local address 127.0.0.1:45156)
0812 19:02:54.118635 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:54.118711 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 19:02:54.124361 19157 catalog_manager.cc:4584] TS 22449ae8d44b4d519d535157b5dcb8e0 (127.29.112.193:45507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73395b703d954a17bb13b137fcbc75e6: Network error: Client connection negotiation failed: client connection to 127.29.112.193:45507: connect: Connection refused (error 111)
I20260812 19:02:54.130242 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 19578
I20260812 19:02:54.139524 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 19136
2026-08-12T19:02:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2307 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T19:02:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:02:54Z Disabled control of system clock
I20260812 19:02:54.173053 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:43333
--webserver_interface=127.29.112.254
--webserver_port=0
--builtin_ntp_servers=127.29.112.212:44329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:43333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:02:54.278146 19802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:54.278362 19802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:54.278393 19802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:54.281977 19802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:02:54.282015 19802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:54.282029 19802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:54.282043 19802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:02:54.282058 19802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:02:54.286288 19802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:43333
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:43333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:54.287134 19802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:54.288349 19802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:02:54.296135 19802 server_base.cc:1034] running on GCE node
W20260812 19:02:54.296059 19813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:54.296157 19811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:54.296060 19808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:54.296836 19802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:54.297765 19802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:54.298961 19802 hybrid_clock.cc:648] HybridClock initialized: now 1786561374298946 us; error 28 us; skew 500 ppm
I20260812 19:02:54.300616 19802 webserver.cc:460] Webserver started at http://127.29.112.254:35355/ using document root <none> and password file <none>
I20260812 19:02:54.301113 19802 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:54.301172 19802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:54.301389 19802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:54.303332 19802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "252e2501aaa1417c9ebd56097c57a15c"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "17ef70d494fbe14360c6740515bb735c"
server_key_iv: "29d05771a01e6618bcc07aad2816512c"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.303834 19802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "252e2501aaa1417c9ebd56097c57a15c"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "17ef70d494fbe14360c6740515bb735c"
server_key_iv: "29d05771a01e6618bcc07aad2816512c"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.307034 19802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:02:54.309297 19822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:54.310483 19802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:02:54.310571 19802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "252e2501aaa1417c9ebd56097c57a15c"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "17ef70d494fbe14360c6740515bb735c"
server_key_iv: "29d05771a01e6618bcc07aad2816512c"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.310683 19802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:54.327098 19802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:54.327713 19802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:54.327908 19802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:54.334564 19802 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:43333
I20260812 19:02:54.334576 19885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:43333 every 8 connection(s)
I20260812 19:02:54.335489 19802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:02:54.338467 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 19802
I20260812 19:02:54.338591 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:02:54.338691 19886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:54.338932 30147 external_mini_cluster.cc:1428] Setting key 3dc55afebed1cb694aec5e2f3f915976
I20260812 19:02:54.345280 19886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c: Bootstrap starting.
I20260812 19:02:54.347718 19886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:54.348001 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:54.340238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48494 (local address 127.29.112.254:43333)
0812 19:02:54.340847 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:54.340862 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:54.340896 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:02:54.341753 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:54.341771 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:54.342174 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:54.342433 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:54.342740 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.343631 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.344811 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.345244 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.345351 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:54.346249 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:54.346254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:54.346264 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:54.346611 (+ 347us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:54.346670 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:54.346688 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:54.346785 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:54.346994 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 19:02:54.347283 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":180,"threads_started":1}
I20260812 19:02:54.348532 19886 log.cc:824] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:54.350593 19886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c: No bootstrap required, opened a new log
I20260812 19:02:54.353986 19886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252e2501aaa1417c9ebd56097c57a15c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43333 } }
I20260812 19:02:54.354185 19886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:54.354264 19886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 252e2501aaa1417c9ebd56097c57a15c, State: Initialized, Role: FOLLOWER
I20260812 19:02:54.354699 19886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252e2501aaa1417c9ebd56097c57a15c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43333 } }
I20260812 19:02:54.354836 19886 raft_consensus.cc:388] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:02:54.354898 19886 raft_consensus.cc:482] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:02:54.355011 19886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:54.355852 19886 raft_consensus.cc:504] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252e2501aaa1417c9ebd56097c57a15c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43333 } }
I20260812 19:02:54.356220 19886 leader_election.cc:302] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 252e2501aaa1417c9ebd56097c57a15c; no voters:
I20260812 19:02:54.356583 19886 leader_election.cc:288] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:02:54.356756 19894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:54.357023 19894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [term 1 LEADER]: Becoming Leader. State: Replica: 252e2501aaa1417c9ebd56097c57a15c, State: Running, Role: LEADER
I20260812 19:02:54.357353 19894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252e2501aaa1417c9ebd56097c57a15c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43333 } }
I20260812 19:02:54.357753 19886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:02:54.359197 19896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "252e2501aaa1417c9ebd56097c57a15c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252e2501aaa1417c9ebd56097c57a15c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43333 } } }
I20260812 19:02:54.359146 19898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 252e2501aaa1417c9ebd56097c57a15c. Latest consensus state: current_term: 1 leader_uuid: "252e2501aaa1417c9ebd56097c57a15c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252e2501aaa1417c9ebd56097c57a15c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43333 } } }
I20260812 19:02:54.359305 19898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:54.359305 19896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c [sys.catalog]: This master's current role is: LEADER
W20260812 19:02:54.362479 19912 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:02:54.362541 19912 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:02:54.362609 19910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:02:54.363226 19910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:02:54.368177 19910 catalog_manager.cc:1347] Generated new cluster ID: 8eed7419e17d4845be2301f5285a36bc
I20260812 19:02:54.368237 19910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:02:54.393102 19910 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:02:54.394065 19910 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:02:54.406010 19910 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c: Generated new TSK 0
I20260812 19:02:54.406658 19910 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:02:54.415370 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43333
--builtin_ntp_servers=127.29.112.212:44329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:02:54.542745 19916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:54.542987 19916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:54.543025 19916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:54.546953 19916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:54.547005 19916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:54.547083 19916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:02:54.551926 19916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:54.552944 19916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:54.554322 19916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:54.563417 19929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:54.563450 19923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:54.563524 19916 server_base.cc:1034] running on GCE node
W20260812 19:02:54.563421 19924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:54.564214 19916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:54.564899 19916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:54.566056 19916 hybrid_clock.cc:648] HybridClock initialized: now 1786561374566037 us; error 30 us; skew 500 ppm
I20260812 19:02:54.567808 19916 webserver.cc:460] Webserver started at http://127.29.112.193:43145/ using document root <none> and password file <none>
I20260812 19:02:54.568389 19916 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:54.568498 19916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:54.568724 19916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:54.570734 19916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "db31d22c718af85d8477fc1cdca75fc3"
server_key_iv: "f07a63bc6c9cde697a4cc52d3d7abafd"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.571256 19916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "db31d22c718af85d8477fc1cdca75fc3"
server_key_iv: "f07a63bc6c9cde697a4cc52d3d7abafd"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.574800 19916 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:02:54.577018 19935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:54.578225 19916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:02:54.578347 19916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "db31d22c718af85d8477fc1cdca75fc3"
server_key_iv: "f07a63bc6c9cde697a4cc52d3d7abafd"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.578459 19916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:54.611580 19916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:54.612284 19916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:54.612490 19916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:54.613117 19916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:54.614406 19916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:54.614490 19916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:54.614547 19916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:54.614581 19916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:54.623842 19916 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:34751
I20260812 19:02:54.623881 20067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:34751 every 8 connection(s)
I20260812 19:02:54.624666 19916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:02:54.633513 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 19916
I20260812 19:02:54.633605 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:54.626331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:33073 (local address 127.29.112.254:43333)
0812 19:02:54.626476 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:54.626480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:54.627498 (+ 1018us) server_negotiation.cc:407] Connection header received
0812 19:02:54.628905 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:54.628908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:54.628969 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:54.629089 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:54.629972 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.630537 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.631626 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.631873 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.631985 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:54.633072 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:54.633076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:54.633079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:54.633091 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:54.633117 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:54.633125 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:54.633199 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:54.633405 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 19:02:54.633465 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:02:54.633661 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 19:02:54.634016 30147 external_mini_cluster.cc:1428] Setting key f11bf8065ba0d277ae5dd636f68d75e9
I20260812 19:02:54.634594 20071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:54.626664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43333 (local address 127.29.112.193:33073)
0812 19:02:54.627325 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:54.627372 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:54.628700 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:54.629246 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:54.629253 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:54.629332 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:54.629776 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:54.629788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.630660 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.630663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:54.631492 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:54.631501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.632022 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.632026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:54.632180 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:54.632780 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:02:54.632782 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:54.632832 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:02:54.632891 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:54.633213 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:54.633223 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:54.633245 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 19:02:54.633376 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 19:02:54.633673 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":125,"threads_started":1}
I20260812 19:02:54.635864 20069 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43333
I20260812 19:02:54.636092 20069 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:54.636196 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43333
--builtin_ntp_servers=127.29.112.212:44329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:02:54.636655 20069 heartbeater.cc:511] Master 127.29.112.254:43333 requested a full tablet report, sending...
I20260812 19:02:54.638163 19843 ts_manager.cc:194] Registered new tserver with Master: 689a6cbd465b4d8c92f5d9f4a6aeada8 (127.29.112.193:34751)
I20260812 19:02:54.639575 19843 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.193:33073
W20260812 19:02:54.746007 20073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:54.746253 20073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:54.746291 20073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:54.750064 20073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:54.750109 20073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:54.750161 20073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:02:54.754390 20073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:54.755262 20073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:54.756456 20073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:54.765759 20081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:54.765764 20078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:54.765902 20073 server_base.cc:1034] running on GCE node
W20260812 19:02:54.765798 20079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:54.766352 20073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:54.767091 20073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:54.768297 20073 hybrid_clock.cc:648] HybridClock initialized: now 1786561374768284 us; error 30 us; skew 500 ppm
I20260812 19:02:54.777486 20073 webserver.cc:460] Webserver started at http://127.29.112.194:44923/ using document root <none> and password file <none>
I20260812 19:02:54.778115 20073 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:54.778208 20073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:54.778439 20073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:54.780323 20073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "dc87c59623b9446f9250a284cb718e48"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "8fba041ad6ebb21d82c6eb4b869cea38"
server_key_iv: "35ad8962fceaafd7c28add086fb83b5a"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.780951 20073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "dc87c59623b9446f9250a284cb718e48"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "8fba041ad6ebb21d82c6eb4b869cea38"
server_key_iv: "35ad8962fceaafd7c28add086fb83b5a"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.784749 20073 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 19:02:54.787101 20089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:54.788190 20073 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:02:54.788313 20073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "dc87c59623b9446f9250a284cb718e48"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "8fba041ad6ebb21d82c6eb4b869cea38"
server_key_iv: "35ad8962fceaafd7c28add086fb83b5a"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.788427 20073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:54.812014 20073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:54.812646 20073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:54.812829 20073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:54.813429 20073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:54.814478 20073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:54.814544 20073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:54.814608 20073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:54.814654 20073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:54.823071 20073 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:40113
I20260812 19:02:54.823100 20230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:40113 every 8 connection(s)
I20260812 19:02:54.823897 20073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:02:54.832698 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:54.825591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:36991 (local address 127.29.112.254:43333)
0812 19:02:54.825753 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:54.825757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:54.827066 (+ 1309us) server_negotiation.cc:407] Connection header received
0812 19:02:54.828387 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:54.828390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:54.828434 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:54.828512 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:54.829309 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.829942 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.830998 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.831200 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.831274 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:54.832286 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:54.832289 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:54.832291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:54.832303 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:54.832321 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:54.832329 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:54.832387 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:54.832542 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 19:02:54.832591 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 19:02:54.833499 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 20073
I20260812 19:02:54.833628 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 19:02:54.833612 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:54.826072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43333 (local address 127.29.112.194:36991)
0812 19:02:54.826869 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:54.826925 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:54.828214 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:54.828631 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:54.828637 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:54.828713 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:54.829191 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:54.829203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.830065 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.830068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:54.830889 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:54.830907 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:54.831324 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:54.831326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:54.831462 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:54.832029 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:02:54.832031 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:54.832071 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:02:54.832135 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:54.832408 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:54.832417 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:54.832425 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 19:02:54.832529 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 19:02:54.832746 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":143,"threads_started":1}
I20260812 19:02:54.834065 30147 external_mini_cluster.cc:1428] Setting key a5902e30fcc19837a8ecc161acb6c012
I20260812 19:02:54.834969 20231 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43333
I20260812 19:02:54.835201 20231 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:54.835755 20231 heartbeater.cc:511] Master 127.29.112.254:43333 requested a full tablet report, sending...
I20260812 19:02:54.836195 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43333
--builtin_ntp_servers=127.29.112.212:44329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:02:54.836853 19843 ts_manager.cc:194] Registered new tserver with Master: dc87c59623b9446f9250a284cb718e48 (127.29.112.194:40113)
I20260812 19:02:54.837483 19843 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.194:36991
W20260812 19:02:54.943084 20235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:54.943336 20235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:54.943404 20235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:54.947255 20235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:54.947324 20235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:54.947408 20235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:02:54.951961 20235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:54.952909 20235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:54.954587 20235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:54.964330 20240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:54.964342 20243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:54.964342 20241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:54.964738 20235 server_base.cc:1034] running on GCE node
I20260812 19:02:54.965106 20235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:54.965668 20235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:54.966856 20235 hybrid_clock.cc:648] HybridClock initialized: now 1786561374966843 us; error 30 us; skew 500 ppm
I20260812 19:02:54.968657 20235 webserver.cc:460] Webserver started at http://127.29.112.195:43129/ using document root <none> and password file <none>
I20260812 19:02:54.969177 20235 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:54.969262 20235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:54.969470 20235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:54.971454 20235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "e9920bc4f17d4778acb817001453a770"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "30e59a99dda5c9f1b6b67797f03c5d67"
server_key_iv: "120e02a13adec3dd4dbb17901c1bf89e"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.972085 20235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "e9920bc4f17d4778acb817001453a770"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "30e59a99dda5c9f1b6b67797f03c5d67"
server_key_iv: "120e02a13adec3dd4dbb17901c1bf89e"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.975605 20235 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 19:02:54.977890 20251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:54.979017 20235 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 19:02:54.979140 20235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "e9920bc4f17d4778acb817001453a770"
format_stamp: "Formatted at 2026-08-12 19:02:54 on dist-test-slave-6nmv"
server_key: "30e59a99dda5c9f1b6b67797f03c5d67"
server_key_iv: "120e02a13adec3dd4dbb17901c1bf89e"
server_key_version: "encryptionkey@0"
I20260812 19:02:54.979241 20235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:54.996100 20235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:54.996690 20235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:54.996893 20235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:54.997418 20235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:54.998389 20235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:54.998466 20235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:54.998548 20235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:54.998595 20235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:55.006867 20235 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:40017
I20260812 19:02:55.006894 20388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:40017 every 8 connection(s)
I20260812 19:02:55.007681 20235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:02:55.012830 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 20235
I20260812 19:02:55.012959 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 19:02:55.013247 30147 external_mini_cluster.cc:1428] Setting key 1acfb0b3f78fe3db9c9c5dbdda16774d
I20260812 19:02:55.016656 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.009393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:37865 (local address 127.29.112.254:43333)
0812 19:02:55.009537 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:55.009540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:55.010543 (+ 1003us) server_negotiation.cc:407] Connection header received
0812 19:02:55.011787 (+ 1244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:55.011789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:55.011836 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:55.011907 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:55.012760 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.013537 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.014984 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.015251 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.015374 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.016292 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:55.016295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:55.016297 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:55.016308 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:55.016326 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:55.016333 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:55.016388 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:55.016509 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 19:02:55.016554 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:02:55.017472 20391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.009709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43333 (local address 127.29.112.195:37865)
0812 19:02:55.010383 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:55.010422 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:55.011607 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:55.012068 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:55.012075 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:55.012169 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:55.012614 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.012626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.013669 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.013672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.014883 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.014893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.015418 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.015421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.015553 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.016077 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:02:55.016078 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:55.016114 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:02:55.016169 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:55.016398 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:55.016405 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:55.016409 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 19:02:55.016494 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 19:02:55.016717 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":163,"threads_started":1}
I20260812 19:02:55.018486 20389 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43333
I20260812 19:02:55.018689 20389 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:55.019132 20389 heartbeater.cc:511] Master 127.29.112.254:43333 requested a full tablet report, sending...
I20260812 19:02:55.019989 19843 ts_manager.cc:194] Registered new tserver with Master: e9920bc4f17d4778acb817001453a770 (127.29.112.195:40017)
I20260812 19:02:55.020455 19843 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.195:37865
I20260812 19:02:55.026988 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:02:55.032313 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.028819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45356 (local address 127.29.112.254:43333)
0812 19:02:55.028943 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:55.028946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:55.029107 (+ 161us) server_negotiation.cc:407] Connection header received
0812 19:02:55.029190 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:55.029192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:55.029228 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:55.029288 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:55.029515 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.029989 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.030722 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.030924 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.031000 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.031842 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:55.031844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:55.031847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:55.031859 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:55.031883 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:55.031893 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:55.032009 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:55.032097 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 19:02:55.032156 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 19:02:55.034941 19843 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:02:55.037695 19843 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:02:55.051441 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.046544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.29.112.193:34751)
0812 19:02:55.047511 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:55.047520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:55.047536 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:02:55.047595 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:55.047598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:55.047712 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:55.047855 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:55.048043 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.048870 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.050229 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.050727 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.050872 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.051049 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:55.051195 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 19:02:55.051286 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":882,"thread_start_us":134,"threads_started":1}
I20260812 19:02:55.051435 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.046150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:34751 (local address 127.0.0.1:54690)
0812 19:02:55.047216 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:55.047231 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:55.047303 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:55.047833 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:55.047836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:55.047853 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:55.047937 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.047943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.048997 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.049000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.050124 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.050130 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.050859 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.050862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.050972 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.051003 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 19:02:55.051148 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 19:02:55.051283 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260812 19:02:55.051854 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.045289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:40017 (local address 127.0.0.1:33842)
0812 19:02:55.046024 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:55.046088 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:55.046219 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:55.046977 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:55.046980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:55.047012 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:55.047138 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.047144 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.048634 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.048636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.049907 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.049923 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.051222 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.051227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.051430 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.051521 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 19:02:55.051603 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 19:02:55.051707 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":180,"spinlock_wait_cycles":9344,"thread_start_us":106,"threads_started":1}
I20260812 19:02:55.051986 20408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.045618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.29.112.194:40113)
0812 19:02:55.046511 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:55.046526 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:55.046546 (+ 20us) server_negotiation.cc:407] Connection header received
0812 19:02:55.046617 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:55.046621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:55.046794 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:55.046976 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:55.047323 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.048469 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.050087 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.050812 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.050954 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.051131 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:55.051698 (+ 567us) server_negotiation.cc:299] Negotiation successful
0812 19:02:55.051791 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":796,"thread_start_us":98,"threads_started":1}
I20260812 19:02:55.052217 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.045398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33842 (local address 127.29.112.195:40017)
0812 19:02:55.045998 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:55.046008 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:55.046354 (+ 346us) server_negotiation.cc:407] Connection header received
0812 19:02:55.046446 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:55.046451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:55.046636 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:55.046794 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:55.047259 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.048464 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.050031 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.050812 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.050954 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.051131 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:55.051811 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 19:02:55.051917 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":112,"threads_started":1}
I20260812 19:02:55.052450 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.045443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:40113 (local address 127.0.0.1:49952)
0812 19:02:55.046153 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:55.046164 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:55.046249 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:55.046968 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:55.046974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:55.047008 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:55.047170 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.047179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.048600 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.048603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.049953 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.049963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.051222 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.051227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.051451 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.051529 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 19:02:55.051603 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 19:02:55.052291 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":669,"spinlock_wait_cycles":4480,"thread_start_us":101,"threads_started":1}
I20260812 19:02:55.054448 20316 tablet_service.cc:1467] Processing CreateTablet for tablet d65e81ed06fa4ba997e48460413c21b8 (DEFAULT_TABLE table=test-table [id=e009f8ec06e74baa9542ed59599e3660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:55.054450 19992 tablet_service.cc:1467] Processing CreateTablet for tablet d65e81ed06fa4ba997e48460413c21b8 (DEFAULT_TABLE table=test-table [id=e009f8ec06e74baa9542ed59599e3660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:55.055387 20316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d65e81ed06fa4ba997e48460413c21b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:55.055439 19992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d65e81ed06fa4ba997e48460413c21b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:55.057356 20150 tablet_service.cc:1467] Processing CreateTablet for tablet d65e81ed06fa4ba997e48460413c21b8 (DEFAULT_TABLE table=test-table [id=e009f8ec06e74baa9542ed59599e3660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:55.058346 20150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d65e81ed06fa4ba997e48460413c21b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:55.062259 20412 tablet_bootstrap.cc:492] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770: Bootstrap starting.
I20260812 19:02:55.063167 20411 tablet_bootstrap.cc:492] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8: Bootstrap starting.
I20260812 19:02:55.064359 20413 tablet_bootstrap.cc:492] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: Bootstrap starting.
I20260812 19:02:55.064705 20412 tablet_bootstrap.cc:654] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:55.065255 20411 tablet_bootstrap.cc:654] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:55.065670 20412 log.cc:824] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:55.066134 20413 tablet_bootstrap.cc:654] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:55.066143 20411 log.cc:824] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:55.066917 20413 log.cc:824] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:55.067945 20412 tablet_bootstrap.cc:492] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770: No bootstrap required, opened a new log
I20260812 19:02:55.068146 20412 ts_tablet_manager.cc:1397] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:02:55.068562 20413 tablet_bootstrap.cc:492] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: No bootstrap required, opened a new log
I20260812 19:02:55.068729 20413 ts_tablet_manager.cc:1397] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 19:02:55.071038 20411 tablet_bootstrap.cc:492] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8: No bootstrap required, opened a new log
I20260812 19:02:55.071214 20411 ts_tablet_manager.cc:1397] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 19:02:55.071624 20413 raft_consensus.cc:348] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9920bc4f17d4778acb817001453a770" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40017 } } peers { permanent_uuid: "dc87c59623b9446f9250a284cb718e48" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40113 } } peers { permanent_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34751 } }
I20260812 19:02:55.071848 20413 raft_consensus.cc:374] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:55.071909 20413 raft_consensus.cc:729] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc87c59623b9446f9250a284cb718e48, State: Initialized, Role: FOLLOWER
I20260812 19:02:55.071960 20412 raft_consensus.cc:348] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9920bc4f17d4778acb817001453a770" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40017 } } peers { permanent_uuid: "dc87c59623b9446f9250a284cb718e48" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40113 } } peers { permanent_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34751 } }
I20260812 19:02:55.072208 20412 raft_consensus.cc:374] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:55.072273 20412 raft_consensus.cc:729] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9920bc4f17d4778acb817001453a770, State: Initialized, Role: FOLLOWER
I20260812 19:02:55.072351 20413 consensus_queue.cc:260] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9920bc4f17d4778acb817001453a770" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40017 } } peers { permanent_uuid: "dc87c59623b9446f9250a284cb718e48" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40113 } } peers { permanent_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34751 } }
I20260812 19:02:55.072714 20412 consensus_queue.cc:260] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9920bc4f17d4778acb817001453a770" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40017 } } peers { permanent_uuid: "dc87c59623b9446f9250a284cb718e48" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40113 } } peers { permanent_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34751 } }
I20260812 19:02:55.073002 20231 heartbeater.cc:503] Master 127.29.112.254:43333 was elected leader, sending a full tablet report...
I20260812 19:02:55.073271 20413 ts_tablet_manager.cc:1428] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 19:02:55.073617 20412 ts_tablet_manager.cc:1428] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 19:02:55.073941 20389 heartbeater.cc:503] Master 127.29.112.254:43333 was elected leader, sending a full tablet report...
W20260812 19:02:55.074752 20232 tablet.cc:2369] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:02:55.075923 20411 raft_consensus.cc:348] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9920bc4f17d4778acb817001453a770" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40017 } } peers { permanent_uuid: "dc87c59623b9446f9250a284cb718e48" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40113 } } peers { permanent_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34751 } }
I20260812 19:02:55.076184 20411 raft_consensus.cc:374] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:55.076257 20411 raft_consensus.cc:729] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 689a6cbd465b4d8c92f5d9f4a6aeada8, State: Initialized, Role: FOLLOWER
I20260812 19:02:55.076668 20411 consensus_queue.cc:260] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9920bc4f17d4778acb817001453a770" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40017 } } peers { permanent_uuid: "dc87c59623b9446f9250a284cb718e48" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40113 } } peers { permanent_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34751 } }
I20260812 19:02:55.077425 20069 heartbeater.cc:503] Master 127.29.112.254:43333 was elected leader, sending a full tablet report...
I20260812 19:02:55.077641 20411 ts_tablet_manager.cc:1428] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260812 19:02:55.125784 20070 tablet.cc:2369] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:02:55.161159 20418 raft_consensus.cc:482] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:55.161375 20418 raft_consensus.cc:504] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9920bc4f17d4778acb817001453a770" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40017 } } peers { permanent_uuid: "dc87c59623b9446f9250a284cb718e48" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40113 } } peers { permanent_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34751 } }
I20260812 19:02:55.162612 20418 leader_election.cc:288] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc87c59623b9446f9250a284cb718e48 (127.29.112.194:40113), 689a6cbd465b4d8c92f5d9f4a6aeada8 (127.29.112.193:34751)
I20260812 19:02:55.167166 20391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.162845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:40113 (local address 127.29.112.195:33461)
0812 19:02:55.162976 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:55.162992 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:55.163089 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:55.163474 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:55.163478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:55.163508 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:55.163629 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.163639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.164650 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.164653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.165824 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.165872 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.166698 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.166701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.166832 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.166861 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 19:02:55.166938 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 19:02:55.167013 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 19:02:55.167199 20420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.163168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:34751 (local address 127.29.112.195:60967)
0812 19:02:55.163574 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:55.163588 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:55.163678 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:55.163910 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:55.163912 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:55.163925 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:55.164017 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.164021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.164880 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.164883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.165996 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:55.166002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.166854 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.166857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:55.166955 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.166973 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 19:02:55.167037 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 19:02:55.167090 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":127,"threads_started":1}
I20260812 19:02:55.167196 20408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.162956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:33461 (local address 127.29.112.194:40113)
0812 19:02:55.163100 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:55.163108 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:55.163133 (+ 25us) server_negotiation.cc:407] Connection header received
0812 19:02:55.163198 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:55.163204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:55.163284 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:55.163388 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:55.163744 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.164521 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.165985 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.166583 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.166718 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.166816 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:55.166969 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 19:02:55.167066 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:02:55.167263 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.163291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:60967 (local address 127.29.112.193:34751)
0812 19:02:55.163435 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:55.163440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:55.163659 (+ 219us) server_negotiation.cc:407] Connection header received
0812 19:02:55.163758 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:55.163761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:55.163809 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:55.163889 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:55.164118 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.164769 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.166109 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.166707 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.166843 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.166891 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:55.167065 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 19:02:55.167138 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 19:02:55.167840 20013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d65e81ed06fa4ba997e48460413c21b8" candidate_uuid: "e9920bc4f17d4778acb817001453a770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" is_pre_election: true
I20260812 19:02:55.167877 20173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d65e81ed06fa4ba997e48460413c21b8" candidate_uuid: "e9920bc4f17d4778acb817001453a770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc87c59623b9446f9250a284cb718e48" is_pre_election: true
I20260812 19:02:55.168154 20013 raft_consensus.cc:2454] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9920bc4f17d4778acb817001453a770 in term 0.
I20260812 19:02:55.168164 20173 raft_consensus.cc:2454] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9920bc4f17d4778acb817001453a770 in term 0.
I20260812 19:02:55.168627 20254 leader_election.cc:302] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc87c59623b9446f9250a284cb718e48, e9920bc4f17d4778acb817001453a770; no voters:
I20260812 19:02:55.168902 20418 raft_consensus.cc:2781] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:02:55.169010 20418 raft_consensus.cc:482] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:02:55.169068 20418 raft_consensus.cc:3037] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:55.170179 20418 raft_consensus.cc:504] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9920bc4f17d4778acb817001453a770" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40017 } } peers { permanent_uuid: "dc87c59623b9446f9250a284cb718e48" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40113 } } peers { permanent_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34751 } }
I20260812 19:02:55.170514 20418 leader_election.cc:288] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [CANDIDATE]: Term 1 election: Requested vote from peers dc87c59623b9446f9250a284cb718e48 (127.29.112.194:40113), 689a6cbd465b4d8c92f5d9f4a6aeada8 (127.29.112.193:34751)
I20260812 19:02:55.170786 20173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d65e81ed06fa4ba997e48460413c21b8" candidate_uuid: "e9920bc4f17d4778acb817001453a770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc87c59623b9446f9250a284cb718e48"
I20260812 19:02:55.170903 20173 raft_consensus.cc:3037] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:55.170910 20013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d65e81ed06fa4ba997e48460413c21b8" candidate_uuid: "e9920bc4f17d4778acb817001453a770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8"
I20260812 19:02:55.171041 20013 raft_consensus.cc:3037] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:55.171916 20173 raft_consensus.cc:2454] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9920bc4f17d4778acb817001453a770 in term 1.
I20260812 19:02:55.172152 20013 raft_consensus.cc:2454] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9920bc4f17d4778acb817001453a770 in term 1.
I20260812 19:02:55.172230 20254 leader_election.cc:302] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc87c59623b9446f9250a284cb718e48, e9920bc4f17d4778acb817001453a770; no voters:
I20260812 19:02:55.172468 20418 raft_consensus.cc:2781] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:55.172719 20418 raft_consensus.cc:686] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 1 LEADER]: Becoming Leader. State: Replica: e9920bc4f17d4778acb817001453a770, State: Running, Role: LEADER
I20260812 19:02:55.173041 20418 consensus_queue.cc:237] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9920bc4f17d4778acb817001453a770" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40017 } } peers { permanent_uuid: "dc87c59623b9446f9250a284cb718e48" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40113 } } peers { permanent_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34751 } }
I20260812 19:02:55.176003 19844 catalog_manager.cc:5495] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9920bc4f17d4778acb817001453a770 (127.29.112.195). New cstate: current_term: 1 leader_uuid: "e9920bc4f17d4778acb817001453a770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9920bc4f17d4778acb817001453a770" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 40017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc87c59623b9446f9250a284cb718e48" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20260812 19:02:55.201696 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.198601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.29.112.195:40017)
0812 19:02:55.198710 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:55.198713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:55.198724 (+ 11us) server_negotiation.cc:407] Connection header received
0812 19:02:55.198845 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:55.198847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:55.198892 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:55.198971 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:02:55.199224 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.199656 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.200385 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:55.200598 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:55.200688 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:55.201004 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:02:55.201348 (+ 344us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:02:55.201454 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:55.201551 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 19:02:55.201586 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 19:02:55.207020 20173 raft_consensus.cc:1261] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48 [term 1 FOLLOWER]: Refusing update from remote peer e9920bc4f17d4778acb817001453a770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:55.207026 20013 raft_consensus.cc:1261] T d65e81ed06fa4ba997e48460413c21b8 P 689a6cbd465b4d8c92f5d9f4a6aeada8 [term 1 FOLLOWER]: Refusing update from remote peer e9920bc4f17d4778acb817001453a770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:55.207664 20421 consensus_queue.cc:1035] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc87c59623b9446f9250a284cb718e48" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:55.207803 20418 consensus_queue.cc:1035] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "689a6cbd465b4d8c92f5d9f4a6aeada8" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:55.215031 20426 mvcc.cc:205] Tried to move back new op lower bound from 7317755392841703424 to 7317755392711237632. Current Snapshot: MvccSnapshot[applied={T|T < 7317755392841703424}]
I20260812 19:02:55.216121 20425 mvcc.cc:205] Tried to move back new op lower bound from 7317755392841703424 to 7317755392711237632. Current Snapshot: MvccSnapshot[applied={T|T < 7317755392841703424}]
I20260812 19:02:55.222857 19843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:55.222986 19843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:55.225176 19843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 252e2501aaa1417c9ebd56097c57a15c: Sending DeleteTablet for 3 replicas of tablet d65e81ed06fa4ba997e48460413c21b8
I20260812 19:02:55.225839 20316 tablet_service.cc:1514] Processing DeleteTablet for tablet d65e81ed06fa4ba997e48460413c21b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:55 UTC) from {username='slave'} at 127.0.0.1:33842
I20260812 19:02:55.226082 19992 tablet_service.cc:1514] Processing DeleteTablet for tablet d65e81ed06fa4ba997e48460413c21b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:55 UTC) from {username='slave'} at 127.0.0.1:54690
I20260812 19:02:55.226076 20150 tablet_service.cc:1514] Processing DeleteTablet for tablet d65e81ed06fa4ba997e48460413c21b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:55 UTC) from {username='slave'} at 127.0.0.1:49952
I20260812 19:02:55.226400 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 19916
I20260812 19:02:55.226488 20437 tablet_replica.cc:330] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: stopping tablet replica
I20260812 19:02:55.226504 20436 tablet_replica.cc:330] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770: stopping tablet replica
I20260812 19:02:55.226756 20437 raft_consensus.cc:2229] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:02:55.226755 20436 raft_consensus.cc:2229] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:02:55.227051 20437 raft_consensus.cc:2258] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:55.227104 20436 raft_consensus.cc:2258] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:55.229562 20437 ts_tablet_manager.cc:1905] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:55.232489 20437 ts_tablet_manager.cc:1918] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:55.232591 20437 log.cc:1196] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/d65e81ed06fa4ba997e48460413c21b8
I20260812 19:02:55.232960 20437 ts_tablet_manager.cc:1939] T d65e81ed06fa4ba997e48460413c21b8 P dc87c59623b9446f9250a284cb718e48: Deleting consensus metadata
I20260812 19:02:55.233702 20436 ts_tablet_manager.cc:1905] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:55.234182 19825 catalog_manager.cc:4841] TS dc87c59623b9446f9250a284cb718e48 (127.29.112.194:40113): tablet d65e81ed06fa4ba997e48460413c21b8 (table test-table [id=e009f8ec06e74baa9542ed59599e3660]) successfully deleted
I20260812 19:02:55.237229 20436 ts_tablet_manager.cc:1918] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:55.237529 20436 log.cc:1196] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/d65e81ed06fa4ba997e48460413c21b8
W20260812 19:02:55.237614 19825 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:34751 (error 108)
I20260812 19:02:55.237715 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 20073
I20260812 19:02:55.239768 20436 ts_tablet_manager.cc:1939] T d65e81ed06fa4ba997e48460413c21b8 P e9920bc4f17d4778acb817001453a770: Deleting consensus metadata
I20260812 19:02:55.240940 19823 catalog_manager.cc:4841] TS e9920bc4f17d4778acb817001453a770 (127.29.112.195:40017): tablet d65e81ed06fa4ba997e48460413c21b8 (table test-table [id=e009f8ec06e74baa9542ed59599e3660]) successfully deleted
I20260812 19:02:55.241660 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:55.241368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:34751 (local address 127.0.0.1:54692)
0812 19:02:55.241510 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:55.241578 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:34751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 19:02:55.241892 19825 catalog_manager.cc:4584] TS 689a6cbd465b4d8c92f5d9f4a6aeada8 (127.29.112.193:34751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d65e81ed06fa4ba997e48460413c21b8: Network error: Client connection negotiation failed: client connection to 127.29.112.193:34751: connect: Connection refused (error 111)
I20260812 19:02:55.246160 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 20235
I20260812 19:02:55.253547 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 19802
2026-08-12T19:02:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1113 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:02:55 dist-test-slave-6nmv krb5kdc[20443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:02:55 dist-test-slave-6nmv krb5kdc[20443](info): set up 2 sockets
Aug 12 19:02:55 dist-test-slave-6nmv krb5kdc[20443](info): commencing operation
krb5kdc: starting...
W20260812 19:02:56.411862 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:02:56 dist-test-slave-6nmv krb5kdc[20443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:02:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:02:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:02:56.528622 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:35425
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:35425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:56.636675 20460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:56.636893 20460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:56.636992 20460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:56.640831 20460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:02:56.640880 20460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:56.640898 20460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:56.640914 20460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:02:56.640929 20460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:02:56.646394 20460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:35425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:35425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:56.647352 20460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:56.648324 20460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:56.656723 20470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:56.656747 20465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:56.656855 20460 server_base.cc:1034] running on GCE node
W20260812 19:02:56.656777 20467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:56.657547 20460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:56.658619 20460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:56.659773 20460 hybrid_clock.cc:648] HybridClock initialized: now 1786561376659749 us; error 38 us; skew 500 ppm
Aug 12 19:02:56 dist-test-slave-6nmv krb5kdc[20443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561376, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:56.662575 20460 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:02:56.663511 20460 webserver.cc:460] Webserver started at http://127.29.112.254:44855/ using document root <none> and password file <none>
I20260812 19:02:56.664201 20460 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:56.664281 20460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:56.664505 20460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:56.666509 20460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "152f218d85bb4661bc4b34d93a6b2890"
format_stamp: "Formatted at 2026-08-12 19:02:56 on dist-test-slave-6nmv"
server_key: "73a504cac86910814391fc2bdaf50658"
server_key_iv: "cc41971ff8d40e03e3342caabd138006"
server_key_version: "encryptionkey@0"
I20260812 19:02:56.667022 20460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "152f218d85bb4661bc4b34d93a6b2890"
format_stamp: "Formatted at 2026-08-12 19:02:56 on dist-test-slave-6nmv"
server_key: "73a504cac86910814391fc2bdaf50658"
server_key_iv: "cc41971ff8d40e03e3342caabd138006"
server_key_version: "encryptionkey@0"
I20260812 19:02:56.670476 20460 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:02:56.672823 20478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:56.674152 20460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:02:56.674250 20460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "152f218d85bb4661bc4b34d93a6b2890"
format_stamp: "Formatted at 2026-08-12 19:02:56 on dist-test-slave-6nmv"
server_key: "73a504cac86910814391fc2bdaf50658"
server_key_iv: "cc41971ff8d40e03e3342caabd138006"
server_key_version: "encryptionkey@0"
I20260812 19:02:56.674329 20460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:56.683172 20460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:56.685925 20460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:56.686067 20460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:56.693022 20460 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:35425
I20260812 19:02:56.693041 20540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:35425 every 8 connection(s)
I20260812 19:02:56.694018 20460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:02:56.696667 20541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:56.702275 20541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890: Bootstrap starting.
I20260812 19:02:56.704077 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 20460
I20260812 19:02:56.704208 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:02:56.704486 30147 external_mini_cluster.cc:1428] Setting key 598f2ee0e2433aab69bbd601f0df2c72
I20260812 19:02:56.704641 20541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:56.705528 20541 log.cc:824] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:56.707626 20541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890: No bootstrap required, opened a new log
I20260812 19:02:56.711782 20541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152f218d85bb4661bc4b34d93a6b2890" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35425 } }
I20260812 19:02:56.712052 20541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:56.712118 20541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152f218d85bb4661bc4b34d93a6b2890, State: Initialized, Role: FOLLOWER
I20260812 19:02:56.712532 20541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152f218d85bb4661bc4b34d93a6b2890" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35425 } }
I20260812 19:02:56.712652 20541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:02:56.712733 20541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:02:56.712828 20541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 19:02:56 dist-test-slave-6nmv krb5kdc[20443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:56.713838 20541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152f218d85bb4661bc4b34d93a6b2890" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35425 } }
I20260812 19:02:56.714202 20541 leader_election.cc:302] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 152f218d85bb4661bc4b34d93a6b2890; no voters:
I20260812 19:02:56.714601 20541 leader_election.cc:288] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:02:56.714679 20548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:56.714859 20548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [term 1 LEADER]: Becoming Leader. State: Replica: 152f218d85bb4661bc4b34d93a6b2890, State: Running, Role: LEADER
I20260812 19:02:56.715133 20548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152f218d85bb4661bc4b34d93a6b2890" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35425 } }
I20260812 19:02:56.715790 20541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:02:56.717269 20550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 152f218d85bb4661bc4b34d93a6b2890. Latest consensus state: current_term: 1 leader_uuid: "152f218d85bb4661bc4b34d93a6b2890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152f218d85bb4661bc4b34d93a6b2890" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35425 } } }
I20260812 19:02:56.717394 20550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:56.718039 20549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "152f218d85bb4661bc4b34d93a6b2890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152f218d85bb4661bc4b34d93a6b2890" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35425 } } }
I20260812 19:02:56.718161 20549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:56.718361 20566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:02:56.720052 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:56.705558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43414 (local address 127.29.112.254:35425)
0812 19:02:56.706008 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:56.706022 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:56.706054 (+ 32us) server_negotiation.cc:407] Connection header received
0812 19:02:56.706842 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:56.706882 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:56.707264 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:56.707605 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:56.708396 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:56.709893 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:56.710680 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:56.711346 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:56.711503 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:56.714514 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:56.714519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:56.714530 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:56.714554 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:56.716667 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:56.717178 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:56.717184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:56.717189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:56.717341 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:56.717714 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:56.717717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:56.717719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:56.718263 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:56.718463 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:56.718908 (+ 445us) server_negotiation.cc:299] Negotiation successful
0812 19:02:56.719183 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":130,"threads_started":1}
I20260812 19:02:56.721334 20566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:02:56.726281 20566 catalog_manager.cc:1347] Generated new cluster ID: 59cd9d5b4af04facb42c99df0ced1871
I20260812 19:02:56.726351 20566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:02:56.732368 20566 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:02:56.733422 20566 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:02:56.739507 20566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890: Generated new TSK 0
I20260812 19:02:56.739992 20566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
Entry for principal kudu/127.29.112.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab.
I20260812 19:02:56.777143 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:35425
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:56.907054 20577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:56.907279 20577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:56.907352 20577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:56.911448 20577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:56.911496 20577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:56.911592 20577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:02:56.916592 20577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.193.keytab
--principal=kudu/127.29.112.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:35425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:56.917568 20577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:56.918540 20577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:56.926867 20587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:56.926863 20584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:56.926877 20583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:56.927135 20577 server_base.cc:1034] running on GCE node
I20260812 19:02:56.927631 20577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:56.928211 20577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:56.929359 20577 hybrid_clock.cc:648] HybridClock initialized: now 1786561376929343 us; error 29 us; skew 500 ppm
Aug 12 19:02:56 dist-test-slave-6nmv krb5kdc[20443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561376, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:56.931996 20577 init.cc:377] Logged in from keytab as kudu/127.29.112.193@KRBTEST.COM (short username kudu)
I20260812 19:02:56.932790 20577 webserver.cc:460] Webserver started at http://127.29.112.193:45463/ using document root <none> and password file <none>
I20260812 19:02:56.933279 20577 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:56.933333 20577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:56.933555 20577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:56.935365 20577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "625a267f0a914310880654cb13f9c14a"
format_stamp: "Formatted at 2026-08-12 19:02:56 on dist-test-slave-6nmv"
server_key: "c79bef4425bd272efbb772269ca9dd7e"
server_key_iv: "aa8c9c6b4a0703db8a399662f99b0b06"
server_key_version: "encryptionkey@0"
I20260812 19:02:56.935868 20577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "625a267f0a914310880654cb13f9c14a"
format_stamp: "Formatted at 2026-08-12 19:02:56 on dist-test-slave-6nmv"
server_key: "c79bef4425bd272efbb772269ca9dd7e"
server_key_iv: "aa8c9c6b4a0703db8a399662f99b0b06"
server_key_version: "encryptionkey@0"
I20260812 19:02:56.939254 20577 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:02:56.941457 20595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:56.942575 20577 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:02:56.942684 20577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "625a267f0a914310880654cb13f9c14a"
format_stamp: "Formatted at 2026-08-12 19:02:56 on dist-test-slave-6nmv"
server_key: "c79bef4425bd272efbb772269ca9dd7e"
server_key_iv: "aa8c9c6b4a0703db8a399662f99b0b06"
server_key_version: "encryptionkey@0"
I20260812 19:02:56.942772 20577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:56.964740 20577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:56.967923 20577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:56.968133 20577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:56.968719 20577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:56.969511 20577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:56.969568 20577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:56.969642 20577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:56.969676 20577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:56.978582 20577 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:34569
I20260812 19:02:56.978655 20736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:34569 every 8 connection(s)
I20260812 19:02:56.979434 20577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:02:56.984424 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 20577
I20260812 19:02:56.984555 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 19:02:56.984841 30147 external_mini_cluster.cc:1428] Setting key edb1c56e0f970d04d19d580cb683f754
Aug 12 19:02:56 dist-test-slave-6nmv krb5kdc[20443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561376, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.193@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:56.994117 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:56.981095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:56931 (local address 127.29.112.254:35425)
0812 19:02:56.981232 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:56.981236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:56.982126 (+ 890us) server_negotiation.cc:407] Connection header received
0812 19:02:56.983404 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:56.983407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:56.983463 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:56.983550 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:56.984668 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:56.985260 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:56.986594 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:56.986842 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:56.986965 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:56.990033 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:56.990039 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:56.990045 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:56.990079 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:56.991893 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:56.992383 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:56.992389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:56.992394 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:56.992509 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:56.992841 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:56.992847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:56.992850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:56.993125 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:56.993260 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:56.993737 (+ 477us) server_negotiation.cc:299] Negotiation successful
0812 19:02:56.993922 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 19:02:56.994822 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:56.981429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:35425 (local address 127.29.112.193:56931)
0812 19:02:56.981957 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:56.982010 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:56.983192 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:56.983715 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:56.983722 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:56.984045 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:56.984469 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:56.984505 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:56.985379 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:56.985382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:56.986468 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:56.986478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:56.987016 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:56.987021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:56.987187 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:56.987834 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:56.987850 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:56.989876 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:56.992011 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:56.992015 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:56.992026 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:56.992266 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:56.992613 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:56.992614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:56.992616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:56.992739 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:56.993251 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:56.993255 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:56.993478 (+ 223us) client_negotiation.cc:769] Sending connection context
0812 19:02:56.993611 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 19:02:56.993967 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":126,"threads_started":1}
WARNING: no policy specified for kudu/127.29.112.194@KRBTEST.COM; defaulting to no policy
I20260812 19:02:56.996018 20737 heartbeater.cc:347] Connected to a master server at 127.29.112.254:35425
I20260812 19:02:56.996228 20737 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:56.996692 20737 heartbeater.cc:511] Master 127.29.112.254:35425 requested a full tablet report, sending...
I20260812 19:02:56.998245 20501 ts_manager.cc:194] Registered new tserver with Master: 625a267f0a914310880654cb13f9c14a (127.29.112.193:34569)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.194@KRBTEST.COM" created.
I20260812 19:02:56.999480 20501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.193@KRBTEST.COM'} at 127.29.112.193:56931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
Entry for principal kudu/127.29.112.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab.
I20260812 19:02:57.013931 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:35425
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:57.116236 20742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:57.116453 20742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:57.116518 20742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:57.120433 20742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:57.120481 20742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:57.120533 20742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:02:57.125108 20742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.194.keytab
--principal=kudu/127.29.112.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:35425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:57.125890 20742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:57.126765 20742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:57.135923 20751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:57.136101 20753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:57.136147 20742 server_base.cc:1034] running on GCE node
W20260812 19:02:57.136242 20750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:57.136659 20742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:57.137264 20742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:57.138414 20742 hybrid_clock.cc:648] HybridClock initialized: now 1786561377138401 us; error 25 us; skew 500 ppm
Aug 12 19:02:57 dist-test-slave-6nmv krb5kdc[20443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561377, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:57.140975 20742 init.cc:377] Logged in from keytab as kudu/127.29.112.194@KRBTEST.COM (short username kudu)
I20260812 19:02:57.141870 20742 webserver.cc:460] Webserver started at http://127.29.112.194:42481/ using document root <none> and password file <none>
I20260812 19:02:57.142381 20742 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:57.142442 20742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:57.142645 20742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:57.144480 20742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "746c05375866424d82de4845a0154c4f"
format_stamp: "Formatted at 2026-08-12 19:02:57 on dist-test-slave-6nmv"
server_key: "df67edd7807158b2141019379fd4c4f4"
server_key_iv: "badb227de60a28a60d187e61bda3ace5"
server_key_version: "encryptionkey@0"
I20260812 19:02:57.145015 20742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "746c05375866424d82de4845a0154c4f"
format_stamp: "Formatted at 2026-08-12 19:02:57 on dist-test-slave-6nmv"
server_key: "df67edd7807158b2141019379fd4c4f4"
server_key_iv: "badb227de60a28a60d187e61bda3ace5"
server_key_version: "encryptionkey@0"
I20260812 19:02:57.148576 20742 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 19:02:57.150801 20761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:57.151966 20742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:02:57.152081 20742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "746c05375866424d82de4845a0154c4f"
format_stamp: "Formatted at 2026-08-12 19:02:57 on dist-test-slave-6nmv"
server_key: "df67edd7807158b2141019379fd4c4f4"
server_key_iv: "badb227de60a28a60d187e61bda3ace5"
server_key_version: "encryptionkey@0"
I20260812 19:02:57.152174 20742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:57.173853 20742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:57.176501 20742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:57.176685 20742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:57.177276 20742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:57.178254 20742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:57.178318 20742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:57.178380 20742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:57.178416 20742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:57.186568 20742 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:35761
I20260812 19:02:57.186592 20897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:35761 every 8 connection(s)
I20260812 19:02:57.187386 20742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:02:57.189982 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 20742
I20260812 19:02:57.190120 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 19:02:57.190436 30147 external_mini_cluster.cc:1428] Setting key f54dc7fdaa5b72983e3a331db5feeede
Aug 12 19:02:57 dist-test-slave-6nmv krb5kdc[20443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561377, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.194@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.195@KRBTEST.COM; defaulting to no policy
I20260812 19:02:57.201877 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.189035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:37113 (local address 127.29.112.254:35425)
0812 19:02:57.189170 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:57.189175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:57.190096 (+ 921us) server_negotiation.cc:407] Connection header received
0812 19:02:57.191570 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:57.191572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:57.191622 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:57.191733 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:57.192868 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.193696 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.195443 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.195657 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.195731 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.198537 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:57.198540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:57.198543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:57.198565 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:57.199950 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:57.200416 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:57.200419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:57.200420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:57.200496 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:57.200806 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:57.200809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:57.200811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:57.201004 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:57.201100 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:57.201540 (+ 440us) server_negotiation.cc:299] Negotiation successful
0812 19:02:57.201685 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:02:57.202555 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.189428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:35425 (local address 127.29.112.194:37113)
0812 19:02:57.189926 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:57.189995 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:57.191368 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:57.191922 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:57.191931 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:57.192280 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:57.192706 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.192718 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.193868 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.193870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.195310 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.195324 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.195799 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.195806 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.196034 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.196618 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:57.196642 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:57.198360 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:57.200058 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:57.200063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:57.200076 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:57.200312 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:57.200584 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:57.200587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:57.200589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:57.200714 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:57.201100 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:57.201107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:57.201317 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 19:02:57.201476 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 19:02:57.201786 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":111,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.195@KRBTEST.COM" created.
I20260812 19:02:57.203634 20898 heartbeater.cc:347] Connected to a master server at 127.29.112.254:35425
I20260812 19:02:57.203866 20898 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:57.204290 20898 heartbeater.cc:511] Master 127.29.112.254:35425 requested a full tablet report, sending...
I20260812 19:02:57.205240 20501 ts_manager.cc:194] Registered new tserver with Master: 746c05375866424d82de4845a0154c4f (127.29.112.194:35761)
I20260812 19:02:57.205744 20501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.194@KRBTEST.COM'} at 127.29.112.194:37113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
Entry for principal kudu/127.29.112.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab.
I20260812 19:02:57.218864 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:35425
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:02:57.326779 20903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:57.327035 20903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:57.327077 20903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:57.330915 20903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:57.330964 20903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:57.331017 20903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:02:57.336054 20903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.195.keytab
--principal=kudu/127.29.112.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:35425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:57.336956 20903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:57.338014 20903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:57.347218 20911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:57.347225 20909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:57.347229 20903 server_base.cc:1034] running on GCE node
W20260812 19:02:57.347239 20913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:57.347796 20903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:57.348392 20903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:57.349573 20903 hybrid_clock.cc:648] HybridClock initialized: now 1786561377349566 us; error 31 us; skew 500 ppm
Aug 12 19:02:57 dist-test-slave-6nmv krb5kdc[20443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561377, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:02:57.352332 20903 init.cc:377] Logged in from keytab as kudu/127.29.112.195@KRBTEST.COM (short username kudu)
I20260812 19:02:57.353097 20903 webserver.cc:460] Webserver started at http://127.29.112.195:45911/ using document root <none> and password file <none>
I20260812 19:02:57.353577 20903 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:57.353631 20903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:57.353780 20903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:57.355784 20903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "f8b660b3b87b436280fd2722471ca390"
format_stamp: "Formatted at 2026-08-12 19:02:57 on dist-test-slave-6nmv"
server_key: "3189f15a24d5525972a26bfd7ff5e2e6"
server_key_iv: "20f3fc3f24361d3dd9faf22870738865"
server_key_version: "encryptionkey@0"
I20260812 19:02:57.356292 20903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "f8b660b3b87b436280fd2722471ca390"
format_stamp: "Formatted at 2026-08-12 19:02:57 on dist-test-slave-6nmv"
server_key: "3189f15a24d5525972a26bfd7ff5e2e6"
server_key_iv: "20f3fc3f24361d3dd9faf22870738865"
server_key_version: "encryptionkey@0"
I20260812 19:02:57.359716 20903 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260812 19:02:57.361871 20922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:57.363036 20903 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:02:57.363122 20903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "f8b660b3b87b436280fd2722471ca390"
format_stamp: "Formatted at 2026-08-12 19:02:57 on dist-test-slave-6nmv"
server_key: "3189f15a24d5525972a26bfd7ff5e2e6"
server_key_iv: "20f3fc3f24361d3dd9faf22870738865"
server_key_version: "encryptionkey@0"
I20260812 19:02:57.363194 20903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:57.383086 20903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:57.385694 20903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:57.385891 20903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:57.386371 20903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:57.387158 20903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:57.387204 20903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:57.387241 20903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:57.387256 20903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:57.396216 20903 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:36815
I20260812 19:02:57.396257 21061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:36815 every 8 connection(s)
I20260812 19:02:57.397034 20903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:02:57.404985 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 20903
I20260812 19:02:57.405081 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 19:02:57.405355 30147 external_mini_cluster.cc:1428] Setting key 1ba3db700eff7873588841d755dfc8cc
Aug 12 19:02:57 dist-test-slave-6nmv krb5kdc[20443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561377, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.195@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:02:57.410038 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.398748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:59507 (local address 127.29.112.254:35425)
0812 19:02:57.398907 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:57.398912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:57.399668 (+ 756us) server_negotiation.cc:407] Connection header received
0812 19:02:57.400858 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:57.400861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:57.400907 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:57.400979 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:57.402120 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.402603 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.403671 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.403859 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.403927 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.406506 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:57.406512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:57.406516 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:57.406544 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:57.408249 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:57.408718 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:57.408721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:57.408722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:57.408797 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:57.409064 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:57.409066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:57.409067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:57.409221 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:57.409293 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:57.409775 (+ 482us) server_negotiation.cc:299] Negotiation successful
0812 19:02:57.409927 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 19:02:57.410696 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.399076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:35425 (local address 127.29.112.195:59507)
0812 19:02:57.399516 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:57.399558 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:57.400658 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:57.401133 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:57.401139 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:57.401476 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:57.401984 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.402000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.402724 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.402726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.403568 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.403577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.403972 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.403974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.404097 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.404697 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:02:57.404713 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:57.406350 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:57.408341 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:57.408345 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:57.408371 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:57.408605 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:57.408876 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:02:57.408877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:02:57.408879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:02:57.408993 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:02:57.409300 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:57.409303 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:57.409489 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 19:02:57.409692 (+ 203us) client_negotiation.cc:240] Negotiation successful
0812 19:02:57.409958 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":111,"threads_started":1}
I20260812 19:02:57.411863 21062 heartbeater.cc:347] Connected to a master server at 127.29.112.254:35425
I20260812 19:02:57.412082 21062 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:57.412559 21062 heartbeater.cc:511] Master 127.29.112.254:35425 requested a full tablet report, sending...
I20260812 19:02:57.413563 20501 ts_manager.cc:194] Registered new tserver with Master: f8b660b3b87b436280fd2722471ca390 (127.29.112.195:36815)
I20260812 19:02:57.414166 20501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.195@KRBTEST.COM'} at 127.29.112.195:59507
I20260812 19:02:57.419034 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:02:57.427906 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.420867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.29.112.254:35425)
0812 19:02:57.420998 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:57.421002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:57.421190 (+ 188us) server_negotiation.cc:407] Connection header received
0812 19:02:57.421322 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:57.421325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:57.421367 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:57.421445 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:57.422202 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.422696 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.423521 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.423738 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.423832 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.425042 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:57.425045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:57.425048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:02:57.425077 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:57.426254 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:57.426690 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:57.426693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:57.426694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:57.426766 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:02:57.427060 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:02:57.427063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:02:57.427065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:02:57.427273 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:57.427412 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:57.427640 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 19:02:57.427751 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 19:02:57.430630 20501 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:02:57.433576 20501 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:02:57.448062 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.442100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:35761 (local address 127.0.0.1:56614)
0812 19:02:57.442755 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:57.442770 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:57.442859 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:57.443771 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:57.443774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:57.443785 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:57.443881 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.443888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.445373 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.445376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.446624 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.446635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.447525 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.447529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.447636 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.447674 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 19:02:57.447799 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 19:02:57.447927 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":141,"thread_start_us":88,"threads_started":1}
I20260812 19:02:57.448069 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.442177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56614 (local address 127.29.112.194:35761)
0812 19:02:57.443345 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:57.443354 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:57.443389 (+ 35us) server_negotiation.cc:407] Connection header received
0812 19:02:57.443452 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:57.443456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:57.443641 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:57.443802 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:57.443973 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.445186 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.446758 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.447398 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.447529 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.447661 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:57.447829 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 19:02:57.447920 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1079,"thread_start_us":154,"threads_started":1}
I20260812 19:02:57.448503 21075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.441907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46394 (local address 127.29.112.195:36815)
0812 19:02:57.443098 (+ 1191us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:57.443110 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:57.443144 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:02:57.443214 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:57.443218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:57.443430 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:57.443590 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:57.444023 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.444746 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.446581 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.447475 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.447615 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.447749 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:57.448262 (+ 513us) server_negotiation.cc:299] Negotiation successful
0812 19:02:57.448360 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1093,"thread_start_us":158,"threads_started":1}
I20260812 19:02:57.448417 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.442393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:34569 (local address 127.0.0.1:45910)
0812 19:02:57.443017 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:57.443036 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:57.443106 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:57.443594 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:57.443600 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:57.443623 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:57.443781 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.443789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.445370 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.445373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.446775 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.446784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.447887 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.447891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.447968 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.448074 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 19:02:57.448165 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 19:02:57.448263 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":217,"thread_start_us":86,"threads_started":1}
I20260812 19:02:57.448873 21076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.441693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:36815 (local address 127.0.0.1:46394)
0812 19:02:57.442526 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:57.442576 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:57.442697 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:57.443745 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:57.443750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:57.443769 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:57.443897 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.443902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.444879 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.444882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.446444 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.446456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.447828 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.447836 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.447952 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.448073 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 19:02:57.448169 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 19:02:57.448710 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":506,"thread_start_us":69,"threads_started":1}
I20260812 19:02:57.449332 21081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.442691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45910 (local address 127.29.112.193:34569)
0812 19:02:57.443093 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:57.443110 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:57.443144 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:02:57.443213 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:57.443218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:57.443431 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:57.443590 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:57.443949 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.445186 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.446902 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.447747 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.448509 (+ 762us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.449025 (+ 516us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:57.449084 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 19:02:57.449183 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":95,"threads_started":1}
I20260812 19:02:57.451452 20980 tablet_service.cc:1467] Processing CreateTablet for tablet 6b70a3cf69e14742a5c0c9133bd9f25a (DEFAULT_TABLE table=test-table [id=a19f5ee399cb41c7ad4b53a4204ca42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:57.451637 20817 tablet_service.cc:1467] Processing CreateTablet for tablet 6b70a3cf69e14742a5c0c9133bd9f25a (DEFAULT_TABLE table=test-table [id=a19f5ee399cb41c7ad4b53a4204ca42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:57.451637 20657 tablet_service.cc:1467] Processing CreateTablet for tablet 6b70a3cf69e14742a5c0c9133bd9f25a (DEFAULT_TABLE table=test-table [id=a19f5ee399cb41c7ad4b53a4204ca42c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:57.452381 20980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b70a3cf69e14742a5c0c9133bd9f25a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:57.452615 20817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b70a3cf69e14742a5c0c9133bd9f25a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:57.452615 20657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b70a3cf69e14742a5c0c9133bd9f25a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:57.459007 21082 tablet_bootstrap.cc:492] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390: Bootstrap starting.
I20260812 19:02:57.459990 21084 tablet_bootstrap.cc:492] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a: Bootstrap starting.
I20260812 19:02:57.460093 21083 tablet_bootstrap.cc:492] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f: Bootstrap starting.
I20260812 19:02:57.461091 21082 tablet_bootstrap.cc:654] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:57.461994 21084 tablet_bootstrap.cc:654] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:57.461994 21083 tablet_bootstrap.cc:654] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:57.462241 21082 log.cc:824] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:57.463227 21083 log.cc:824] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:57.463227 21084 log.cc:824] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:57.464202 21082 tablet_bootstrap.cc:492] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390: No bootstrap required, opened a new log
I20260812 19:02:57.464411 21082 ts_tablet_manager.cc:1397] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:02:57.465052 21083 tablet_bootstrap.cc:492] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f: No bootstrap required, opened a new log
I20260812 19:02:57.465052 21084 tablet_bootstrap.cc:492] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a: No bootstrap required, opened a new log
I20260812 19:02:57.465233 21084 ts_tablet_manager.cc:1397] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 19:02:57.465233 21083 ts_tablet_manager.cc:1397] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 19:02:57.467638 21082 raft_consensus.cc:348] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b660b3b87b436280fd2722471ca390" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36815 } } peers { permanent_uuid: "746c05375866424d82de4845a0154c4f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 35761 } } peers { permanent_uuid: "625a267f0a914310880654cb13f9c14a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34569 } }
I20260812 19:02:57.467869 21082 raft_consensus.cc:374] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:57.467929 21082 raft_consensus.cc:729] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8b660b3b87b436280fd2722471ca390, State: Initialized, Role: FOLLOWER
I20260812 19:02:57.468385 21082 consensus_queue.cc:260] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b660b3b87b436280fd2722471ca390" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36815 } } peers { permanent_uuid: "746c05375866424d82de4845a0154c4f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 35761 } } peers { permanent_uuid: "625a267f0a914310880654cb13f9c14a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34569 } }
I20260812 19:02:57.469055 21062 heartbeater.cc:503] Master 127.29.112.254:35425 was elected leader, sending a full tablet report...
I20260812 19:02:57.469128 21084 raft_consensus.cc:348] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b660b3b87b436280fd2722471ca390" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36815 } } peers { permanent_uuid: "746c05375866424d82de4845a0154c4f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 35761 } } peers { permanent_uuid: "625a267f0a914310880654cb13f9c14a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34569 } }
I20260812 19:02:57.469128 21083 raft_consensus.cc:348] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b660b3b87b436280fd2722471ca390" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36815 } } peers { permanent_uuid: "746c05375866424d82de4845a0154c4f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 35761 } } peers { permanent_uuid: "625a267f0a914310880654cb13f9c14a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34569 } }
I20260812 19:02:57.469353 21083 raft_consensus.cc:374] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:57.469353 21084 raft_consensus.cc:374] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:57.469390 21082 ts_tablet_manager.cc:1428] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 19:02:57.469401 21083 raft_consensus.cc:729] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 746c05375866424d82de4845a0154c4f, State: Initialized, Role: FOLLOWER
I20260812 19:02:57.469411 21084 raft_consensus.cc:729] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625a267f0a914310880654cb13f9c14a, State: Initialized, Role: FOLLOWER
I20260812 19:02:57.469856 21084 consensus_queue.cc:260] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b660b3b87b436280fd2722471ca390" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36815 } } peers { permanent_uuid: "746c05375866424d82de4845a0154c4f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 35761 } } peers { permanent_uuid: "625a267f0a914310880654cb13f9c14a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34569 } }
I20260812 19:02:57.469856 21083 consensus_queue.cc:260] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b660b3b87b436280fd2722471ca390" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36815 } } peers { permanent_uuid: "746c05375866424d82de4845a0154c4f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 35761 } } peers { permanent_uuid: "625a267f0a914310880654cb13f9c14a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34569 } }
I20260812 19:02:57.470597 20898 heartbeater.cc:503] Master 127.29.112.254:35425 was elected leader, sending a full tablet report...
I20260812 19:02:57.470732 21084 ts_tablet_manager.cc:1428] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 19:02:57.470732 21083 ts_tablet_manager.cc:1428] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 19:02:57.471022 20737 heartbeater.cc:503] Master 127.29.112.254:35425 was elected leader, sending a full tablet report...
W20260812 19:02:57.480500 20738 tablet.cc:2369] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:02:57.562237 21090 raft_consensus.cc:482] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:57.562441 21090 raft_consensus.cc:504] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b660b3b87b436280fd2722471ca390" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36815 } } peers { permanent_uuid: "746c05375866424d82de4845a0154c4f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 35761 } } peers { permanent_uuid: "625a267f0a914310880654cb13f9c14a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34569 } }
I20260812 19:02:57.563617 21090 leader_election.cc:288] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8b660b3b87b436280fd2722471ca390 (127.29.112.195:36815), 746c05375866424d82de4845a0154c4f (127.29.112.194:35761)
I20260812 19:02:57.568572 21075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.563906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:59701 (local address 127.29.112.195:36815)
0812 19:02:57.564040 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:57.564044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:57.564202 (+ 158us) server_negotiation.cc:407] Connection header received
0812 19:02:57.564333 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:57.564336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:57.564396 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:57.564480 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:57.564886 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.565614 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.567327 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.567971 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.568104 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.568172 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:57.568330 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 19:02:57.568439 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:02:57.568578 21092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.563768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:36815 (local address 127.29.112.193:59701)
0812 19:02:57.564108 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:57.564122 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:57.564226 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:57.564499 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:57.564502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:57.564524 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:57.564772 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.564779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.565723 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.565725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.567180 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.567191 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.568086 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.568089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.568178 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.568199 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 19:02:57.568303 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 19:02:57.568397 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":204,"spinlock_wait_cycles":896,"thread_start_us":91,"threads_started":1}
I20260812 19:02:57.568746 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.564032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:36169 (local address 127.29.112.194:35761)
0812 19:02:57.564175 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:57.564182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:57.564241 (+ 59us) server_negotiation.cc:407] Connection header received
0812 19:02:57.564332 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:57.564335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:57.564391 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:57.564481 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:57.564889 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.565611 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.567539 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.568182 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.568305 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.568368 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:57.568527 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 19:02:57.568602 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 19:02:57.569183 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.563887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:35761 (local address 127.29.112.193:36169)
0812 19:02:57.564164 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:57.564176 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:57.564246 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:57.564499 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:57.564502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:57.564529 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:57.564772 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.564779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.565719 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.565722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.567179 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:57.567191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.568304 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.568308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:57.568413 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.568432 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 19:02:57.568501 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 19:02:57.568550 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":109,"spinlock_wait_cycles":6784,"thread_start_us":104,"threads_started":1}
I20260812 19:02:57.569267 21005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b70a3cf69e14742a5c0c9133bd9f25a" candidate_uuid: "625a267f0a914310880654cb13f9c14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b660b3b87b436280fd2722471ca390" is_pre_election: true
I20260812 19:02:57.569619 21005 raft_consensus.cc:2454] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 625a267f0a914310880654cb13f9c14a in term 0.
I20260812 19:02:57.569907 20841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b70a3cf69e14742a5c0c9133bd9f25a" candidate_uuid: "625a267f0a914310880654cb13f9c14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "746c05375866424d82de4845a0154c4f" is_pre_election: true
I20260812 19:02:57.570179 20841 raft_consensus.cc:2454] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 625a267f0a914310880654cb13f9c14a in term 0.
I20260812 19:02:57.570175 20603 leader_election.cc:302] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 625a267f0a914310880654cb13f9c14a, f8b660b3b87b436280fd2722471ca390; no voters:
I20260812 19:02:57.570456 21090 raft_consensus.cc:2781] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:02:57.570541 21090 raft_consensus.cc:482] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:02:57.570603 21090 raft_consensus.cc:3037] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:57.571673 21090 raft_consensus.cc:504] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b660b3b87b436280fd2722471ca390" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36815 } } peers { permanent_uuid: "746c05375866424d82de4845a0154c4f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 35761 } } peers { permanent_uuid: "625a267f0a914310880654cb13f9c14a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34569 } }
I20260812 19:02:57.572052 21090 leader_election.cc:288] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [CANDIDATE]: Term 1 election: Requested vote from peers f8b660b3b87b436280fd2722471ca390 (127.29.112.195:36815), 746c05375866424d82de4845a0154c4f (127.29.112.194:35761)
I20260812 19:02:57.572360 21005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b70a3cf69e14742a5c0c9133bd9f25a" candidate_uuid: "625a267f0a914310880654cb13f9c14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b660b3b87b436280fd2722471ca390"
I20260812 19:02:57.572502 21005 raft_consensus.cc:3037] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:57.572513 20841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b70a3cf69e14742a5c0c9133bd9f25a" candidate_uuid: "625a267f0a914310880654cb13f9c14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "746c05375866424d82de4845a0154c4f"
I20260812 19:02:57.572634 20841 raft_consensus.cc:3037] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:57.573539 21005 raft_consensus.cc:2454] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 625a267f0a914310880654cb13f9c14a in term 1.
I20260812 19:02:57.573747 20841 raft_consensus.cc:2454] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 625a267f0a914310880654cb13f9c14a in term 1.
I20260812 19:02:57.573846 20603 leader_election.cc:302] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 625a267f0a914310880654cb13f9c14a, f8b660b3b87b436280fd2722471ca390; no voters:
I20260812 19:02:57.574019 21090 raft_consensus.cc:2781] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:57.574270 21090 raft_consensus.cc:686] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 1 LEADER]: Becoming Leader. State: Replica: 625a267f0a914310880654cb13f9c14a, State: Running, Role: LEADER
I20260812 19:02:57.574610 21090 consensus_queue.cc:237] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b660b3b87b436280fd2722471ca390" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36815 } } peers { permanent_uuid: "746c05375866424d82de4845a0154c4f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 35761 } } peers { permanent_uuid: "625a267f0a914310880654cb13f9c14a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34569 } }
I20260812 19:02:57.577929 20497 catalog_manager.cc:5495] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a reported cstate change: term changed from 0 to 1, leader changed from <none> to 625a267f0a914310880654cb13f9c14a (127.29.112.193). New cstate: current_term: 1 leader_uuid: "625a267f0a914310880654cb13f9c14a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b660b3b87b436280fd2722471ca390" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "746c05375866424d82de4845a0154c4f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 35761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "625a267f0a914310880654cb13f9c14a" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 34569 } health_report { overall_health: HEALTHY } } }
I20260812 19:02:57.598243 21081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.595061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45914 (local address 127.29.112.193:34569)
0812 19:02:57.595181 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:57.595184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:57.595197 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:02:57.595333 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:57.595335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:57.595391 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:57.595479 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:02:57.595684 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.596159 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.596920 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.597147 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.597220 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.597461 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:02:57.597936 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:02:57.598028 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:57.598085 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 19:02:57.598134 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:02:57.603900 20841 raft_consensus.cc:1261] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f [term 1 FOLLOWER]: Refusing update from remote peer 625a267f0a914310880654cb13f9c14a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:57.603900 21005 raft_consensus.cc:1261] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390 [term 1 FOLLOWER]: Refusing update from remote peer 625a267f0a914310880654cb13f9c14a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:57.604557 21090 consensus_queue.cc:1035] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8b660b3b87b436280fd2722471ca390" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 36815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:57.604760 21095 consensus_queue.cc:1035] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [LEADER]: Connected to new peer: Peer: permanent_uuid: "746c05375866424d82de4845a0154c4f" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 35761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:57.612962 21100 mvcc.cc:205] Tried to move back new op lower bound from 7317755402659045376 to 7317755402548543488. Current Snapshot: MvccSnapshot[applied={T|T < 7317755402659045376}]
I20260812 19:02:57.613152 21101 mvcc.cc:205] Tried to move back new op lower bound from 7317755402659045376 to 7317755402548543488. Current Snapshot: MvccSnapshot[applied={T|T < 7317755402659045376}]
I20260812 19:02:57.620802 20497 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:57.620971 20497 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:57.623374 20497 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 152f218d85bb4661bc4b34d93a6b2890: Sending DeleteTablet for 3 replicas of tablet 6b70a3cf69e14742a5c0c9133bd9f25a
I20260812 19:02:57.624174 20817 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b70a3cf69e14742a5c0c9133bd9f25a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:57 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:56614
I20260812 19:02:57.624220 20980 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b70a3cf69e14742a5c0c9133bd9f25a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:57 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:46394
I20260812 19:02:57.624290 20657 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b70a3cf69e14742a5c0c9133bd9f25a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:57 UTC) from {username='kudu', principal='kudu/127.29.112.254@KRBTEST.COM'} at 127.0.0.1:45910
I20260812 19:02:57.624749 21114 tablet_replica.cc:330] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a: stopping tablet replica
I20260812 19:02:57.624991 21113 tablet_replica.cc:330] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390: stopping tablet replica
I20260812 19:02:57.625044 21114 raft_consensus.cc:2229] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:02:57.624782 21111 tablet_replica.cc:330] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f: stopping tablet replica
I20260812 19:02:57.625078 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 20577
I20260812 19:02:57.625428 21111 raft_consensus.cc:2229] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:02:57.625434 21114 raft_consensus.cc:2258] T 6b70a3cf69e14742a5c0c9133bd9f25a P 625a267f0a914310880654cb13f9c14a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:57.625849 21113 raft_consensus.cc:2229] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:02:57.626216 21113 raft_consensus.cc:2258] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:57.626724 21111 raft_consensus.cc:2258] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:57.628733 21113 ts_tablet_manager.cc:1905] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:57.628733 21111 ts_tablet_manager.cc:1905] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:57.631675 21111 ts_tablet_manager.cc:1918] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:57.631788 21111 log.cc:1196] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/6b70a3cf69e14742a5c0c9133bd9f25a
I20260812 19:02:57.632094 21111 ts_tablet_manager.cc:1939] T 6b70a3cf69e14742a5c0c9133bd9f25a P 746c05375866424d82de4845a0154c4f: Deleting consensus metadata
I20260812 19:02:57.632465 21113 ts_tablet_manager.cc:1918] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:57.632575 21113 log.cc:1196] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/6b70a3cf69e14742a5c0c9133bd9f25a
I20260812 19:02:57.632916 21113 ts_tablet_manager.cc:1939] T 6b70a3cf69e14742a5c0c9133bd9f25a P f8b660b3b87b436280fd2722471ca390: Deleting consensus metadata
I20260812 19:02:57.633235 20481 catalog_manager.cc:4841] TS 746c05375866424d82de4845a0154c4f (127.29.112.194:35761): tablet 6b70a3cf69e14742a5c0c9133bd9f25a (table test-table [id=a19f5ee399cb41c7ad4b53a4204ca42c]) successfully deleted
I20260812 19:02:57.634128 20482 catalog_manager.cc:4841] TS f8b660b3b87b436280fd2722471ca390 (127.29.112.195:36815): tablet 6b70a3cf69e14742a5c0c9133bd9f25a (table test-table [id=a19f5ee399cb41c7ad4b53a4204ca42c]) successfully deleted
W20260812 19:02:57.637056 20482 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:34569 (error 108)
I20260812 19:02:57.637902 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 20742
I20260812 19:02:57.638131 21076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.637767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:34569 (local address 127.0.0.1:45920)
0812 19:02:57.637930 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:57.638031 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260812 19:02:57.639206 20482 catalog_manager.cc:4584] TS 625a267f0a914310880654cb13f9c14a (127.29.112.193:34569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b70a3cf69e14742a5c0c9133bd9f25a: Network error: Client connection negotiation failed: client connection to 127.29.112.193:34569: connect: Connection refused (error 111)
I20260812 19:02:57.646342 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 20903
I20260812 19:02:57.655423 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 20460
2026-08-12T19:02:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2402 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T19:02:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:02:57Z Disabled control of system clock
I20260812 19:02:57.688395 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:43701
--webserver_interface=127.29.112.254
--webserver_port=0
--builtin_ntp_servers=127.29.112.212:44835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:43701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:02:57.797895 21119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:57.798164 21119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:57.798198 21119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:57.801770 21119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:02:57.801828 21119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:57.801842 21119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:57.801856 21119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:02:57.801869 21119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:02:57.806231 21119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:43701
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:43701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:57.807103 21119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:57.808338 21119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:57.816607 21126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:57.816715 21127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:57.816797 21129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:57.817112 21119 server_base.cc:1034] running on GCE node
I20260812 19:02:57.817618 21119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:57.818708 21119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:57.819886 21119 hybrid_clock.cc:648] HybridClock initialized: now 1786561377819860 us; error 40 us; skew 500 ppm
I20260812 19:02:57.821571 21119 webserver.cc:460] Webserver started at http://127.29.112.254:37315/ using document root <none> and password file <none>
I20260812 19:02:57.822171 21119 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:57.822253 21119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:57.822475 21119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:57.824488 21119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "3c03ee1bedfa4951a81595f2a7d69344"
format_stamp: "Formatted at 2026-08-12 19:02:57 on dist-test-slave-6nmv"
server_key: "89e83f3a488dcd5a6ae3f75365c2fa62"
server_key_iv: "1fb89a003018514076f7a647ef10f43d"
server_key_version: "encryptionkey@0"
I20260812 19:02:57.825105 21119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "3c03ee1bedfa4951a81595f2a7d69344"
format_stamp: "Formatted at 2026-08-12 19:02:57 on dist-test-slave-6nmv"
server_key: "89e83f3a488dcd5a6ae3f75365c2fa62"
server_key_iv: "1fb89a003018514076f7a647ef10f43d"
server_key_version: "encryptionkey@0"
I20260812 19:02:57.828765 21119 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 19:02:57.831003 21135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:57.832139 21119 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:02:57.832278 21119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "3c03ee1bedfa4951a81595f2a7d69344"
format_stamp: "Formatted at 2026-08-12 19:02:57 on dist-test-slave-6nmv"
server_key: "89e83f3a488dcd5a6ae3f75365c2fa62"
server_key_iv: "1fb89a003018514076f7a647ef10f43d"
server_key_version: "encryptionkey@0"
I20260812 19:02:57.832378 21119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:57.858577 21119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:57.859231 21119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:57.859416 21119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:57.866460 21119 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:43701
I20260812 19:02:57.866473 21197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:43701 every 8 connection(s)
I20260812 19:02:57.867384 21119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:02:57.870361 21198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:57.874318 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 21119
I20260812 19:02:57.874451 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:02:57.874763 30147 external_mini_cluster.cc:1428] Setting key a3c2151062a7e77040c9dd794fe8d048
I20260812 19:02:57.876245 21198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344: Bootstrap starting.
I20260812 19:02:57.878537 21198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:57.879240 21198 log.cc:824] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:57.881013 21198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344: No bootstrap required, opened a new log
I20260812 19:02:57.883939 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:57.876172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38714 (local address 127.29.112.254:43701)
0812 19:02:57.876832 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:57.876847 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:57.876882 (+ 35us) server_negotiation.cc:407] Connection header received
0812 19:02:57.877519 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:57.877535 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:57.877996 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:57.878352 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:57.878767 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.879903 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.880787 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:57.881311 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:57.881466 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:57.882369 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:57.882374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:57.882390 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:57.882737 (+ 347us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:57.882784 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:57.882803 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:57.882906 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:57.883039 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 19:02:57.883269 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":161,"threads_started":1}
I20260812 19:02:57.884533 21198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c03ee1bedfa4951a81595f2a7d69344" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43701 } }
I20260812 19:02:57.884725 21198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:57.884757 21198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c03ee1bedfa4951a81595f2a7d69344, State: Initialized, Role: FOLLOWER
I20260812 19:02:57.885236 21198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c03ee1bedfa4951a81595f2a7d69344" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43701 } }
I20260812 19:02:57.885383 21198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:02:57.885466 21198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:02:57.885577 21198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:57.886543 21198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c03ee1bedfa4951a81595f2a7d69344" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43701 } }
I20260812 19:02:57.886852 21198 leader_election.cc:302] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c03ee1bedfa4951a81595f2a7d69344; no voters:
I20260812 19:02:57.887194 21198 leader_election.cc:288] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:02:57.887290 21204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:57.887531 21204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [term 1 LEADER]: Becoming Leader. State: Replica: 3c03ee1bedfa4951a81595f2a7d69344, State: Running, Role: LEADER
I20260812 19:02:57.887831 21204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c03ee1bedfa4951a81595f2a7d69344" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43701 } }
I20260812 19:02:57.888361 21198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:02:57.889465 21206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c03ee1bedfa4951a81595f2a7d69344. Latest consensus state: current_term: 1 leader_uuid: "3c03ee1bedfa4951a81595f2a7d69344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c03ee1bedfa4951a81595f2a7d69344" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43701 } } }
I20260812 19:02:57.889590 21206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:57.889875 21205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c03ee1bedfa4951a81595f2a7d69344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c03ee1bedfa4951a81595f2a7d69344" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43701 } } }
I20260812 19:02:57.889990 21205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344 [sys.catalog]: This master's current role is: LEADER
I20260812 19:02:57.890357 21218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:02:57.892969 21218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:02:57.898126 21218 catalog_manager.cc:1347] Generated new cluster ID: cc814b35670a4d4ea34295d339a0bc42
I20260812 19:02:57.898193 21218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:02:57.907721 21218 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:02:57.908618 21218 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:02:57.921063 21218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344: Generated new TSK 0
I20260812 19:02:57.921622 21218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:02:57.927627 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.193:0
--local_ip_for_outbound_sockets=127.29.112.193
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43701
--builtin_ntp_servers=127.29.112.212:44835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 19:02:58.031867 21229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:58.032124 21229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:58.032188 21229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:58.036079 21229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:58.036139 21229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:58.036214 21229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.193
I20260812 19:02:58.040617 21229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.193
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.193
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:58.041491 21229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:58.042744 21229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:58.051175 21234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:58.051198 21235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:58.051358 21238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:58.051362 21229 server_base.cc:1034] running on GCE node
I20260812 19:02:58.051852 21229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:58.052384 21229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:58.053558 21229 hybrid_clock.cc:648] HybridClock initialized: now 1786561378053545 us; error 23 us; skew 500 ppm
I20260812 19:02:58.055227 21229 webserver.cc:460] Webserver started at http://127.29.112.193:39727/ using document root <none> and password file <none>
I20260812 19:02:58.055742 21229 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:58.055799 21229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:58.055956 21229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:58.057770 21229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/data/instance:
uuid: "edad806e7c9a491fa16863d1bb4eaab3"
format_stamp: "Formatted at 2026-08-12 19:02:58 on dist-test-slave-6nmv"
server_key: "2b059197f444025739172b55e59c8ccb"
server_key_iv: "1183ad5f891ac5a663caaced14bf3c0f"
server_key_version: "encryptionkey@0"
I20260812 19:02:58.058331 21229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance:
uuid: "edad806e7c9a491fa16863d1bb4eaab3"
format_stamp: "Formatted at 2026-08-12 19:02:58 on dist-test-slave-6nmv"
server_key: "2b059197f444025739172b55e59c8ccb"
server_key_iv: "1183ad5f891ac5a663caaced14bf3c0f"
server_key_version: "encryptionkey@0"
I20260812 19:02:58.061450 21229 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 19:02:58.063681 21247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:58.064782 21229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:02:58.064872 21229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/wal
uuid: "edad806e7c9a491fa16863d1bb4eaab3"
format_stamp: "Formatted at 2026-08-12 19:02:58 on dist-test-slave-6nmv"
server_key: "2b059197f444025739172b55e59c8ccb"
server_key_iv: "1183ad5f891ac5a663caaced14bf3c0f"
server_key_version: "encryptionkey@0"
I20260812 19:02:58.064966 21229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:58.082666 21229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:58.083220 21229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:58.083357 21229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:58.083835 21229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:58.084636 21229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:58.084678 21229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:58.084712 21229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:58.084731 21229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:58.093462 21229 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.193:45379
I20260812 19:02:58.093540 21380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.193:45379 every 8 connection(s)
I20260812 19:02:58.094360 21229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/data/info.pb
I20260812 19:02:58.103173 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.096114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.193:37373 (local address 127.29.112.254:43701)
0812 19:02:58.096266 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:58.096271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:58.097386 (+ 1115us) server_negotiation.cc:407] Connection header received
0812 19:02:58.098815 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:58.098817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:58.098858 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:58.098959 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:58.099756 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.100300 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.101373 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.101576 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.101644 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.102804 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:58.102807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:58.102808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:58.102818 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:58.102834 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:58.102840 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:58.102894 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:58.103031 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 19:02:58.103072 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:02:58.103946 21384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.096567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43701 (local address 127.29.112.193:37373)
0812 19:02:58.097246 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:58.097286 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:58.098640 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:58.099090 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:58.099108 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:58.099182 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:58.099613 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.099625 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.100426 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.100429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.101271 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.101281 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.101723 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.101727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.101954 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.102565 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:02:58.102567 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:58.102602 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:02:58.102656 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:58.102919 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:58.102926 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:58.102930 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 19:02:58.103022 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 19:02:58.103231 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":112,"threads_started":1}
I20260812 19:02:58.104113 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 21229
I20260812 19:02:58.104233 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-0/wal/instance
I20260812 19:02:58.104535 30147 external_mini_cluster.cc:1428] Setting key 012fbbbdde6e287d133d017fcfb6a6e1
I20260812 19:02:58.105166 21381 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43701
I20260812 19:02:58.105370 21381 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:58.105952 21381 heartbeater.cc:511] Master 127.29.112.254:43701 requested a full tablet report, sending...
I20260812 19:02:58.106582 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.194:0
--local_ip_for_outbound_sockets=127.29.112.194
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43701
--builtin_ntp_servers=127.29.112.212:44835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:02:58.107338 21155 ts_manager.cc:194] Registered new tserver with Master: edad806e7c9a491fa16863d1bb4eaab3 (127.29.112.193:45379)
I20260812 19:02:58.108717 21155 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.193:37373
W20260812 19:02:58.211050 21385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:58.211297 21385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:58.211333 21385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:58.215317 21385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:58.215368 21385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:58.215416 21385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.194
I20260812 19:02:58.219879 21385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.194
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.194
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:58.220800 21385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:58.222095 21385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:58.231065 21390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:58.231178 21394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:58.231148 21391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:58.231292 21385 server_base.cc:1034] running on GCE node
I20260812 19:02:58.231791 21385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:58.232352 21385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:58.233542 21385 hybrid_clock.cc:648] HybridClock initialized: now 1786561378233521 us; error 32 us; skew 500 ppm
I20260812 19:02:58.235641 21385 webserver.cc:460] Webserver started at http://127.29.112.194:45865/ using document root <none> and password file <none>
I20260812 19:02:58.236253 21385 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:58.236362 21385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:58.236598 21385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:58.238730 21385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/data/instance:
uuid: "8225148be6f248959182539affeae49d"
format_stamp: "Formatted at 2026-08-12 19:02:58 on dist-test-slave-6nmv"
server_key: "385b03ca40725f47e9ef65b79decd2cb"
server_key_iv: "8d61530b4b739317d719f0c8cdcb27e3"
server_key_version: "encryptionkey@0"
I20260812 19:02:58.239313 21385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance:
uuid: "8225148be6f248959182539affeae49d"
format_stamp: "Formatted at 2026-08-12 19:02:58 on dist-test-slave-6nmv"
server_key: "385b03ca40725f47e9ef65b79decd2cb"
server_key_iv: "8d61530b4b739317d719f0c8cdcb27e3"
server_key_version: "encryptionkey@0"
I20260812 19:02:58.243324 21385 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:02:58.246138 21402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:58.247464 21385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 19:02:58.247622 21385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/wal
uuid: "8225148be6f248959182539affeae49d"
format_stamp: "Formatted at 2026-08-12 19:02:58 on dist-test-slave-6nmv"
server_key: "385b03ca40725f47e9ef65b79decd2cb"
server_key_iv: "8d61530b4b739317d719f0c8cdcb27e3"
server_key_version: "encryptionkey@0"
I20260812 19:02:58.247748 21385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:58.265260 21385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:58.265975 21385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:58.266165 21385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:58.266721 21385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:58.267655 21385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:58.267724 21385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:58.267789 21385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:58.267838 21385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:58.276237 21385 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.194:40507
I20260812 19:02:58.276279 21549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.194:40507 every 8 connection(s)
I20260812 19:02:58.277056 21385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/data/info.pb
I20260812 19:02:58.283749 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 21385
I20260812 19:02:58.283906 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/wal/instance
I20260812 19:02:58.284274 30147 external_mini_cluster.cc:1428] Setting key 127129e06a58756dc3c54f9db7c6f8e1
I20260812 19:02:58.286473 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.278786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.194:36973 (local address 127.29.112.254:43701)
0812 19:02:58.278903 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:58.278906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:58.280003 (+ 1097us) server_negotiation.cc:407] Connection header received
0812 19:02:58.281376 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:58.281379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:58.281429 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:58.281492 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:58.282424 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.282922 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.284195 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.284530 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.284646 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.285963 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:58.285966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:58.285969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:58.285982 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:58.286005 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:58.286015 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:58.286092 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:58.286219 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 19:02:58.286273 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 19:02:58.286576 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.195:0
--local_ip_for_outbound_sockets=127.29.112.195
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43701
--builtin_ntp_servers=127.29.112.212:44835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 19:02:58.287192 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.279111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43701 (local address 127.29.112.194:36973)
0812 19:02:58.279788 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:58.279829 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:58.281179 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:58.281652 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:58.281658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:58.281739 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:58.282281 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.282294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.283055 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.283057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.284064 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.284074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.284731 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.284736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.284951 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.285623 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:02:58.285625 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:58.285701 (+ 76us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:02:58.285776 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:58.286096 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:58.286103 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:58.286106 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 19:02:58.286193 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 19:02:58.286387 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":105,"threads_started":1}
I20260812 19:02:58.288316 21550 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43701
I20260812 19:02:58.288553 21550 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:58.289063 21550 heartbeater.cc:511] Master 127.29.112.254:43701 requested a full tablet report, sending...
I20260812 19:02:58.290119 21155 ts_manager.cc:194] Registered new tserver with Master: 8225148be6f248959182539affeae49d (127.29.112.194:40507)
I20260812 19:02:58.290690 21155 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.194:36973
W20260812 19:02:58.393998 21553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:02:58.394249 21553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:02:58.394315 21553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:02:58.398293 21553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:02:58.398360 21553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:02:58.398447 21553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.195
I20260812 19:02:58.403131 21553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.195
--webserver_port=0
--tserver_master_addrs=127.29.112.254:43701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.195
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:02:58.404142 21553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:02:58.405483 21553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:02:58.415134 21558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:02:58.415158 21559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:58.415210 21553 server_base.cc:1034] running on GCE node
W20260812 19:02:58.415151 21563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:02:58.415692 21553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:02:58.416324 21553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:02:58.417541 21553 hybrid_clock.cc:648] HybridClock initialized: now 1786561378417528 us; error 28 us; skew 500 ppm
I20260812 19:02:58.419607 21553 webserver.cc:460] Webserver started at http://127.29.112.195:39705/ using document root <none> and password file <none>
I20260812 19:02:58.420171 21553 fs_manager.cc:359] Metadata directory not provided
I20260812 19:02:58.420258 21553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:02:58.420509 21553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:02:58.422724 21553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/data/instance:
uuid: "1760c181baa34935ac0a3036bbf75c3a"
format_stamp: "Formatted at 2026-08-12 19:02:58 on dist-test-slave-6nmv"
server_key: "1107cc1b567304f518c97e66a5afc48f"
server_key_iv: "5087a84c4d9bb6a4aa0f389bde17d683"
server_key_version: "encryptionkey@0"
I20260812 19:02:58.423264 21553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance:
uuid: "1760c181baa34935ac0a3036bbf75c3a"
format_stamp: "Formatted at 2026-08-12 19:02:58 on dist-test-slave-6nmv"
server_key: "1107cc1b567304f518c97e66a5afc48f"
server_key_iv: "5087a84c4d9bb6a4aa0f389bde17d683"
server_key_version: "encryptionkey@0"
I20260812 19:02:58.426659 21553 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:02:58.428974 21570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:58.430159 21553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 19:02:58.430274 21553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/wal
uuid: "1760c181baa34935ac0a3036bbf75c3a"
format_stamp: "Formatted at 2026-08-12 19:02:58 on dist-test-slave-6nmv"
server_key: "1107cc1b567304f518c97e66a5afc48f"
server_key_iv: "5087a84c4d9bb6a4aa0f389bde17d683"
server_key_version: "encryptionkey@0"
I20260812 19:02:58.430368 21553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:02:58.442479 21553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:02:58.443085 21553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:02:58.443269 21553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:02:58.443768 21553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:02:58.444628 21553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:02:58.444671 21553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:58.444726 21553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:02:58.444762 21553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:02:58.453300 21553 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.195:33539
I20260812 19:02:58.453323 21707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.195:33539 every 8 connection(s)
I20260812 19:02:58.454217 21553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/data/info.pb
I20260812 19:02:58.462889 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.455885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:54757 (local address 127.29.112.254:43701)
0812 19:02:58.456058 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:58.456062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:58.457122 (+ 1060us) server_negotiation.cc:407] Connection header received
0812 19:02:58.458519 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:58.458522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:58.458577 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:58.458645 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:58.459462 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.459960 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.461168 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.461376 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.461462 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.462507 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:58.462510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:58.462511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:58.462522 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:58.462540 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:58.462548 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:58.462599 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:58.462728 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 19:02:58.462772 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 19:02:58.463682 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.456257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:43701 (local address 127.29.112.195:54757)
0812 19:02:58.456957 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:58.456996 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:58.458343 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:58.458794 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:58.458800 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:58.458893 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:02:58.459336 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.459348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.460072 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.460075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.461051 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.461061 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.461529 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.461532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.461669 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.462265 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 19:02:58.462267 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:02:58.462304 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 19:02:58.462364 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:02:58.462614 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:02:58.462620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:02:58.462625 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 19:02:58.462718 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 19:02:58.462980 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":143,"threads_started":1}
I20260812 19:02:58.463940 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 21553
I20260812 19:02:58.464056 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/wal/instance
I20260812 19:02:58.464413 30147 external_mini_cluster.cc:1428] Setting key 3b2de6317c592edf32e3544c8f85eea5
I20260812 19:02:58.464775 21708 heartbeater.cc:347] Connected to a master server at 127.29.112.254:43701
I20260812 19:02:58.464991 21708 heartbeater.cc:464] Registering TS with master...
I20260812 19:02:58.465503 21708 heartbeater.cc:511] Master 127.29.112.254:43701 requested a full tablet report, sending...
I20260812 19:02:58.466568 21155 ts_manager.cc:194] Registered new tserver with Master: 1760c181baa34935ac0a3036bbf75c3a (127.29.112.195:33539)
I20260812 19:02:58.467195 21155 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.195:54757
I20260812 19:02:58.478351 30147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 19:02:58.483965 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.480288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.29.112.254:43701)
0812 19:02:58.480428 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:58.480433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:58.480590 (+ 157us) server_negotiation.cc:407] Connection header received
0812 19:02:58.480764 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:58.480766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:58.480804 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:58.480907 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:02:58.481168 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.481671 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.482514 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.482718 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.482817 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.483512 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:02:58.483515 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:02:58.483518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:02:58.483551 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:02:58.483569 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:02:58.483577 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:02:58.483651 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:58.483754 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 19:02:58.483810 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 19:02:58.486393 21155 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:02:58.489382 21155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:02:58.503046 21722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.497630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45379 (local address 127.0.0.1:42828)
0812 19:02:58.498649 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:58.498662 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:58.498765 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:58.499237 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:58.499239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:58.499252 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:58.499351 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.499354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.500274 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.500280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.501583 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.501591 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.502577 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.502579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.502669 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.502708 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 19:02:58.502807 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 19:02:58.502926 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":24,"thread_start_us":100,"threads_started":1}
I20260812 19:02:58.503082 21725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.498031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.29.112.193:45379)
0812 19:02:58.498528 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:58.498538 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:58.498771 (+ 233us) server_negotiation.cc:407] Connection header received
0812 19:02:58.498843 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:58.498847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:58.498995 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:58.499104 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:58.499481 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.500166 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.501705 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.502472 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.502593 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.502756 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:58.502853 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 19:02:58.502939 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":115,"threads_started":1}
I20260812 19:02:58.503193 21724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.497985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44248 (local address 127.29.112.194:40507)
0812 19:02:58.498429 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:58.498441 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:58.498630 (+ 189us) server_negotiation.cc:407] Connection header received
0812 19:02:58.498692 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:58.498695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:58.498896 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:58.499041 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:58.499500 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.500558 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.501941 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.502601 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.502698 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.502811 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:58.502940 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 19:02:58.503028 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":121,"threads_started":1}
I20260812 19:02:58.503520 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.497498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:40507 (local address 127.0.0.1:44248)
0812 19:02:58.498441 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:58.498453 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:58.498538 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:58.499197 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:58.499199 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:58.499214 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:58.499358 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.499364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.500671 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.500673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.501827 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.501838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.502697 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.502700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.502778 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.502805 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 19:02:58.502859 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 19:02:58.503399 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":569,"thread_start_us":95,"threads_started":1}
I20260812 19:02:58.504078 21720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.497218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.195:33539 (local address 127.0.0.1:36230)
0812 19:02:58.498001 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:58.498052 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:58.498163 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:58.499047 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:58.499052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:58.499071 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:58.499185 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.499191 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.500614 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.500620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.501827 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.501850 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.503617 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.503620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.503821 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.503845 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 19:02:58.503907 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 19:02:58.503973 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"mutex_wait_us":6,"thread_start_us":78,"threads_started":1}
I20260812 19:02:58.504470 21721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.497392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.29.112.195:33539)
0812 19:02:58.498613 (+ 1221us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:58.498624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:58.498645 (+ 21us) server_negotiation.cc:407] Connection header received
0812 19:02:58.498723 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:58.498739 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:58.498912 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:58.499055 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:58.499301 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.500476 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.502868 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.503508 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.503630 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.503766 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:58.504217 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 19:02:58.504310 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1120,"thread_start_us":101,"threads_started":1}
I20260812 19:02:58.506104 21306 tablet_service.cc:1467] Processing CreateTablet for tablet 52cb331cf2e54fa681007cbc8b329171 (DEFAULT_TABLE table=test-table [id=6d0942180b054ec0a57816977ead4af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:58.506105 21466 tablet_service.cc:1467] Processing CreateTablet for tablet 52cb331cf2e54fa681007cbc8b329171 (DEFAULT_TABLE table=test-table [id=6d0942180b054ec0a57816977ead4af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:58.506412 21625 tablet_service.cc:1467] Processing CreateTablet for tablet 52cb331cf2e54fa681007cbc8b329171 (DEFAULT_TABLE table=test-table [id=6d0942180b054ec0a57816977ead4af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 19:02:58.507158 21625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52cb331cf2e54fa681007cbc8b329171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:58.507167 21466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52cb331cf2e54fa681007cbc8b329171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:58.507167 21306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52cb331cf2e54fa681007cbc8b329171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:02:58.513862 21727 tablet_bootstrap.cc:492] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3: Bootstrap starting.
I20260812 19:02:58.514153 21728 tablet_bootstrap.cc:492] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: Bootstrap starting.
I20260812 19:02:58.514657 21726 tablet_bootstrap.cc:492] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a: Bootstrap starting.
I20260812 19:02:58.516093 21728 tablet_bootstrap.cc:654] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:58.516093 21727 tablet_bootstrap.cc:654] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:58.516222 21726 tablet_bootstrap.cc:654] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a: Neither blocks nor log segments found. Creating new log.
I20260812 19:02:58.516880 21726 log.cc:824] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:58.516950 21727 log.cc:824] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:58.516950 21728 log.cc:824] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: Log is configured to *not* fsync() on all Append() calls
I20260812 19:02:58.518929 21726 tablet_bootstrap.cc:492] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a: No bootstrap required, opened a new log
I20260812 19:02:58.518939 21728 tablet_bootstrap.cc:492] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: No bootstrap required, opened a new log
I20260812 19:02:58.519122 21728 ts_tablet_manager.cc:1397] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 19:02:58.519122 21726 ts_tablet_manager.cc:1397] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 19:02:58.519269 21727 tablet_bootstrap.cc:492] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3: No bootstrap required, opened a new log
I20260812 19:02:58.519462 21727 ts_tablet_manager.cc:1397] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:02:58.522564 21727 raft_consensus.cc:348] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1760c181baa34935ac0a3036bbf75c3a" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33539 } } peers { permanent_uuid: "8225148be6f248959182539affeae49d" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40507 } } peers { permanent_uuid: "edad806e7c9a491fa16863d1bb4eaab3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45379 } }
I20260812 19:02:58.522786 21727 raft_consensus.cc:374] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:58.522836 21727 raft_consensus.cc:729] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edad806e7c9a491fa16863d1bb4eaab3, State: Initialized, Role: FOLLOWER
I20260812 19:02:58.523234 21727 consensus_queue.cc:260] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1760c181baa34935ac0a3036bbf75c3a" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33539 } } peers { permanent_uuid: "8225148be6f248959182539affeae49d" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40507 } } peers { permanent_uuid: "edad806e7c9a491fa16863d1bb4eaab3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45379 } }
I20260812 19:02:58.523481 21726 raft_consensus.cc:348] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1760c181baa34935ac0a3036bbf75c3a" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33539 } } peers { permanent_uuid: "8225148be6f248959182539affeae49d" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40507 } } peers { permanent_uuid: "edad806e7c9a491fa16863d1bb4eaab3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45379 } }
I20260812 19:02:58.523481 21728 raft_consensus.cc:348] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1760c181baa34935ac0a3036bbf75c3a" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33539 } } peers { permanent_uuid: "8225148be6f248959182539affeae49d" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40507 } } peers { permanent_uuid: "edad806e7c9a491fa16863d1bb4eaab3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45379 } }
I20260812 19:02:58.523715 21726 raft_consensus.cc:374] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:58.523715 21728 raft_consensus.cc:374] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:02:58.523772 21726 raft_consensus.cc:729] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1760c181baa34935ac0a3036bbf75c3a, State: Initialized, Role: FOLLOWER
I20260812 19:02:58.523965 21381 heartbeater.cc:503] Master 127.29.112.254:43701 was elected leader, sending a full tablet report...
I20260812 19:02:58.524187 21727 ts_tablet_manager.cc:1428] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260812 19:02:58.524220 21726 consensus_queue.cc:260] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1760c181baa34935ac0a3036bbf75c3a" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33539 } } peers { permanent_uuid: "8225148be6f248959182539affeae49d" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40507 } } peers { permanent_uuid: "edad806e7c9a491fa16863d1bb4eaab3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45379 } }
I20260812 19:02:58.523782 21728 raft_consensus.cc:729] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8225148be6f248959182539affeae49d, State: Initialized, Role: FOLLOWER
I20260812 19:02:58.524883 21708 heartbeater.cc:503] Master 127.29.112.254:43701 was elected leader, sending a full tablet report...
I20260812 19:02:58.525141 21726 ts_tablet_manager.cc:1428] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 19:02:58.525249 21728 consensus_queue.cc:260] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1760c181baa34935ac0a3036bbf75c3a" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33539 } } peers { permanent_uuid: "8225148be6f248959182539affeae49d" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40507 } } peers { permanent_uuid: "edad806e7c9a491fa16863d1bb4eaab3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45379 } }
I20260812 19:02:58.526145 21728 ts_tablet_manager.cc:1428] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 19:02:58.526887 21550 heartbeater.cc:503] Master 127.29.112.254:43701 was elected leader, sending a full tablet report...
W20260812 19:02:58.527930 21551 tablet.cc:2369] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:02:58.567701 21735 raft_consensus.cc:482] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:02:58.567898 21735 raft_consensus.cc:504] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1760c181baa34935ac0a3036bbf75c3a" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33539 } } peers { permanent_uuid: "8225148be6f248959182539affeae49d" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40507 } } peers { permanent_uuid: "edad806e7c9a491fa16863d1bb4eaab3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45379 } }
I20260812 19:02:58.569042 21735 leader_election.cc:288] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8225148be6f248959182539affeae49d (127.29.112.194:40507), edad806e7c9a491fa16863d1bb4eaab3 (127.29.112.193:45379)
I20260812 19:02:58.573531 21724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.569436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:51001 (local address 127.29.112.194:40507)
0812 19:02:58.569560 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:58.569565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:58.569658 (+ 93us) server_negotiation.cc:407] Connection header received
0812 19:02:58.569799 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:58.569804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:58.569897 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:58.569978 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:58.570265 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.570838 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.572045 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.572596 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.572715 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.572769 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:58.573324 (+ 555us) server_negotiation.cc:299] Negotiation successful
0812 19:02:58.573399 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:02:58.573522 21725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.569433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.195:58639 (local address 127.29.112.193:45379)
0812 19:02:58.569560 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:58.569565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:58.569931 (+ 366us) server_negotiation.cc:407] Connection header received
0812 19:02:58.570060 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:58.570063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:58.570117 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:58.570200 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 19:02:58.570508 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.571048 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.572110 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.572592 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.572720 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.572769 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:58.573324 (+ 555us) server_negotiation.cc:299] Negotiation successful
0812 19:02:58.573394 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:02:58.573531 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.569328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.194:40507 (local address 127.29.112.195:51001)
0812 19:02:58.569569 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:58.569585 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:58.569694 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:58.570002 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:58.570004 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:58.570024 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:58.570150 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.570160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.570968 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.570972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.571904 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.571914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.573034 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.573040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.573189 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.573217 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 19:02:58.573305 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 19:02:58.573385 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":60}
I20260812 19:02:58.573551 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.569328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45379 (local address 127.29.112.195:58639)
0812 19:02:58.569799 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:58.569813 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:02:58.569970 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:02:58.570222 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:02:58.570227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:02:58.570245 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 19:02:58.570360 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.570366 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.571176 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.571179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.571954 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:02:58.571993 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.573020 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.573023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:02:58.573203 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.573231 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 19:02:58.573307 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 19:02:58.573383 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":57,"thread_start_us":146,"threads_started":1}
I20260812 19:02:58.574221 21329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52cb331cf2e54fa681007cbc8b329171" candidate_uuid: "1760c181baa34935ac0a3036bbf75c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edad806e7c9a491fa16863d1bb4eaab3" is_pre_election: true
I20260812 19:02:58.574249 21489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52cb331cf2e54fa681007cbc8b329171" candidate_uuid: "1760c181baa34935ac0a3036bbf75c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8225148be6f248959182539affeae49d" is_pre_election: true
I20260812 19:02:58.574519 21329 raft_consensus.cc:2454] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1760c181baa34935ac0a3036bbf75c3a in term 0.
I20260812 19:02:58.574527 21489 raft_consensus.cc:2454] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1760c181baa34935ac0a3036bbf75c3a in term 0.
I20260812 19:02:58.574991 21573 leader_election.cc:302] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1760c181baa34935ac0a3036bbf75c3a, edad806e7c9a491fa16863d1bb4eaab3; no voters:
I20260812 19:02:58.575279 21735 raft_consensus.cc:2781] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:02:58.575366 21735 raft_consensus.cc:482] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:02:58.575415 21735 raft_consensus.cc:3037] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:58.576478 21735 raft_consensus.cc:504] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1760c181baa34935ac0a3036bbf75c3a" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33539 } } peers { permanent_uuid: "8225148be6f248959182539affeae49d" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40507 } } peers { permanent_uuid: "edad806e7c9a491fa16863d1bb4eaab3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45379 } }
I20260812 19:02:58.576823 21735 leader_election.cc:288] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [CANDIDATE]: Term 1 election: Requested vote from peers 8225148be6f248959182539affeae49d (127.29.112.194:40507), edad806e7c9a491fa16863d1bb4eaab3 (127.29.112.193:45379)
I20260812 19:02:58.577128 21489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52cb331cf2e54fa681007cbc8b329171" candidate_uuid: "1760c181baa34935ac0a3036bbf75c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8225148be6f248959182539affeae49d"
I20260812 19:02:58.577198 21329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52cb331cf2e54fa681007cbc8b329171" candidate_uuid: "1760c181baa34935ac0a3036bbf75c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edad806e7c9a491fa16863d1bb4eaab3"
I20260812 19:02:58.577244 21489 raft_consensus.cc:3037] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:58.577343 21329 raft_consensus.cc:3037] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:02:58.578418 21489 raft_consensus.cc:2454] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1760c181baa34935ac0a3036bbf75c3a in term 1.
I20260812 19:02:58.578541 21329 raft_consensus.cc:2454] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1760c181baa34935ac0a3036bbf75c3a in term 1.
I20260812 19:02:58.578840 21574 leader_election.cc:302] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1760c181baa34935ac0a3036bbf75c3a, 8225148be6f248959182539affeae49d, edad806e7c9a491fa16863d1bb4eaab3; no voters:
I20260812 19:02:58.579027 21735 raft_consensus.cc:2781] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:02:58.579258 21735 raft_consensus.cc:686] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 1 LEADER]: Becoming Leader. State: Replica: 1760c181baa34935ac0a3036bbf75c3a, State: Running, Role: LEADER
I20260812 19:02:58.579607 21735 consensus_queue.cc:237] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1760c181baa34935ac0a3036bbf75c3a" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33539 } } peers { permanent_uuid: "8225148be6f248959182539affeae49d" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40507 } } peers { permanent_uuid: "edad806e7c9a491fa16863d1bb4eaab3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45379 } }
I20260812 19:02:58.582701 21153 catalog_manager.cc:5495] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1760c181baa34935ac0a3036bbf75c3a (127.29.112.195). New cstate: current_term: 1 leader_uuid: "1760c181baa34935ac0a3036bbf75c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1760c181baa34935ac0a3036bbf75c3a" member_type: VOTER last_known_addr { host: "127.29.112.195" port: 33539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8225148be6f248959182539affeae49d" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edad806e7c9a491fa16863d1bb4eaab3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45379 } health_report { overall_health: UNKNOWN } } }
W20260812 19:02:58.595450 21383 tablet.cc:2369] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:02:58.599964 21721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.596461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.29.112.195:33539)
0812 19:02:58.596585 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 19:02:58.596590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:02:58.596604 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:02:58.596794 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:02:58.596796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:02:58.596849 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:02:58.596958 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:02:58.597206 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.597703 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.598580 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:02:58.598793 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:02:58.598898 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:02:58.599194 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:02:58.599569 (+ 375us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:02:58.599690 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 19:02:58.599750 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 19:02:58.599798 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 19:02:58.606992 21329 raft_consensus.cc:1261] T 52cb331cf2e54fa681007cbc8b329171 P edad806e7c9a491fa16863d1bb4eaab3 [term 1 FOLLOWER]: Refusing update from remote peer 1760c181baa34935ac0a3036bbf75c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:58.606992 21489 raft_consensus.cc:1261] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d [term 1 FOLLOWER]: Refusing update from remote peer 1760c181baa34935ac0a3036bbf75c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:02:58.607640 21738 consensus_queue.cc:1035] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8225148be6f248959182539affeae49d" member_type: VOTER last_known_addr { host: "127.29.112.194" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:58.607847 21735 consensus_queue.cc:1035] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "edad806e7c9a491fa16863d1bb4eaab3" member_type: VOTER last_known_addr { host: "127.29.112.193" port: 45379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:02:58.615603 21742 mvcc.cc:205] Tried to move back new op lower bound from 7317755406766792704 to 7317755406664704000. Current Snapshot: MvccSnapshot[applied={T|T < 7317755406766792704}]
I20260812 19:02:58.624651 21155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:58.624776 21155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 19:02:58.627146 21155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c03ee1bedfa4951a81595f2a7d69344: Sending DeleteTablet for 3 replicas of tablet 52cb331cf2e54fa681007cbc8b329171
I20260812 19:02:58.627897 21466 tablet_service.cc:1514] Processing DeleteTablet for tablet 52cb331cf2e54fa681007cbc8b329171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:58 UTC) from {username='slave'} at 127.0.0.1:44248
I20260812 19:02:58.628048 21306 tablet_service.cc:1514] Processing DeleteTablet for tablet 52cb331cf2e54fa681007cbc8b329171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:58 UTC) from {username='slave'} at 127.0.0.1:42828
I20260812 19:02:58.628160 21625 tablet_service.cc:1514] Processing DeleteTablet for tablet 52cb331cf2e54fa681007cbc8b329171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 19:02:58 UTC) from {username='slave'} at 127.0.0.1:36230
I20260812 19:02:58.628599 21757 tablet_replica.cc:330] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a: stopping tablet replica
I20260812 19:02:58.628888 21757 raft_consensus.cc:2229] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:02:58.629168 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 21229
I20260812 19:02:58.629277 21757 raft_consensus.cc:2258] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:58.633517 21754 tablet_replica.cc:330] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: stopping tablet replica
I20260812 19:02:58.633963 21754 raft_consensus.cc:2229] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:02:58.634330 21754 raft_consensus.cc:2258] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:02:58.635223 21757 ts_tablet_manager.cc:1905] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:58.635866 21754 ts_tablet_manager.cc:1905] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 19:02:58.638509 21754 ts_tablet_manager.cc:1918] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:58.638671 21754 log.cc:1196] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-1/wal/wals/52cb331cf2e54fa681007cbc8b329171
I20260812 19:02:58.638664 21757 ts_tablet_manager.cc:1918] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 19:02:58.638749 21757 log.cc:1196] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a: Deleting WAL directory at /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786561125017040-30147-0/minicluster-data/ts-2/wal/wals/52cb331cf2e54fa681007cbc8b329171
I20260812 19:02:58.639060 21757 ts_tablet_manager.cc:1939] T 52cb331cf2e54fa681007cbc8b329171 P 1760c181baa34935ac0a3036bbf75c3a: Deleting consensus metadata
I20260812 19:02:58.639084 21754 ts_tablet_manager.cc:1939] T 52cb331cf2e54fa681007cbc8b329171 P 8225148be6f248959182539affeae49d: Deleting consensus metadata
I20260812 19:02:58.640197 21138 catalog_manager.cc:4841] TS 1760c181baa34935ac0a3036bbf75c3a (127.29.112.195:33539): tablet 52cb331cf2e54fa681007cbc8b329171 (table test-table [id=6d0942180b054ec0a57816977ead4af6]) successfully deleted
I20260812 19:02:58.640197 21137 catalog_manager.cc:4841] TS 8225148be6f248959182539affeae49d (127.29.112.194:40507): tablet 52cb331cf2e54fa681007cbc8b329171 (table test-table [id=6d0942180b054ec0a57816977ead4af6]) successfully deleted
W20260812 19:02:58.640810 21136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.193:45379 (error 108)
I20260812 19:02:58.641010 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 21385
I20260812 19:02:58.646732 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:02:58.646413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.193:45379 (local address 127.0.0.1:42838)
0812 19:02:58.646567 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 19:02:58.646643 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.193:45379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 19:02:58.646991 21136 catalog_manager.cc:4584] TS edad806e7c9a491fa16863d1bb4eaab3 (127.29.112.193:45379): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52cb331cf2e54fa681007cbc8b329171: Network error: Client connection negotiation failed: client connection to 127.29.112.193:45379: connect: Connection refused (error 111)
I20260812 19:02:58.649780 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 21553
I20260812 19:02:58.656433 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 21119
2026-08-12T19:02:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (999 ms)
[----------] 6 tests from EncryptionPolicyTest (10344 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:02:58 dist-test-slave-6nmv krb5kdc[21763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:02:58 dist-test-slave-6nmv krb5kdc[21763](info): set up 2 sockets
Aug 12 19:02:58 dist-test-slave-6nmv krb5kdc[21763](info): commencing operation
krb5kdc: starting...
W20260812 19:02:59.794190 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:02:59 dist-test-slave-6nmv krb5kdc[21763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:02:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:02:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:02:59.918345 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43339
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:00.031301 21779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:00.031587 21779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:00.031647 21779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:00.035658 21779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:00.035728 21779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:00.035753 21779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:00.035772 21779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:00.035790 21779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:00.040956 21779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43339
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:00.042035 21779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:00.043035 21779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:00.052280 21786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:00.052321 21787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:00.052322 21789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:00.052791 21779 server_base.cc:1034] running on GCE node
I20260812 19:03:00.053359 21779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:00.054545 21779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:00.055745 21779 hybrid_clock.cc:648] HybridClock initialized: now 1786561380055731 us; error 29 us; skew 500 ppm
Aug 12 19:03:00 dist-test-slave-6nmv krb5kdc[21763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:00.064853 21779 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:00.065915 21779 webserver.cc:460] Webserver started at http://127.0.0.1:35527/ using document root <none> and password file <none>
I20260812 19:03:00.066499 21779 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:00.066596 21779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:00.066829 21779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:00.068928 21779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "fdeb6aee5bbc4572941ee60137e2bdbc"
format_stamp: "Formatted at 2026-08-12 19:03:00 on dist-test-slave-6nmv"
server_key: "0729e724a4e65fa2baffc9e3f0de9175"
server_key_iv: "d36871f938c122ae4dcec5338ccb06a2"
server_key_version: "encryptionkey@0"
I20260812 19:03:00.069509 21779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "fdeb6aee5bbc4572941ee60137e2bdbc"
format_stamp: "Formatted at 2026-08-12 19:03:00 on dist-test-slave-6nmv"
server_key: "0729e724a4e65fa2baffc9e3f0de9175"
server_key_iv: "d36871f938c122ae4dcec5338ccb06a2"
server_key_version: "encryptionkey@0"
I20260812 19:03:00.073426 21779 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 19:03:00.076011 21797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:00.077289 21779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 19:03:00.077442 21779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "fdeb6aee5bbc4572941ee60137e2bdbc"
format_stamp: "Formatted at 2026-08-12 19:03:00 on dist-test-slave-6nmv"
server_key: "0729e724a4e65fa2baffc9e3f0de9175"
server_key_iv: "d36871f938c122ae4dcec5338ccb06a2"
server_key_version: "encryptionkey@0"
I20260812 19:03:00.077554 21779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:00.088982 21779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:00.092097 21779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:00.092298 21779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:00.099475 21779 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43339
I20260812 19:03:00.099504 21865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43339 every 8 connection(s)
I20260812 19:03:00.100430 21779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:00.103148 21867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:00.104835 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 21779
I20260812 19:03:00.105002 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:00.105319 30147 external_mini_cluster.cc:1428] Setting key 2d03cd0e8ecc758890d5e3c9daf4bb5f
I20260812 19:03:00.109638 21867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc: Bootstrap starting.
I20260812 19:03:00.112023 21867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:00.112919 21867 log.cc:824] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:03:00 dist-test-slave-6nmv krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 19:03:00.115394 21867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc: No bootstrap required, opened a new log
I20260812 19:03:00.119815 21867 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6aee5bbc4572941ee60137e2bdbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } }
I20260812 19:03:00.120041 21867 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:00.120146 21867 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdeb6aee5bbc4572941ee60137e2bdbc, State: Initialized, Role: FOLLOWER
I20260812 19:03:00.120607 21867 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6aee5bbc4572941ee60137e2bdbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } }
I20260812 19:03:00.120739 21867 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:00.120772 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:00.106958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47843 (local address 127.0.0.1:43339)
0812 19:03:00.107421 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:00.107436 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:00.107470 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:03:00.108107 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:00.108123 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:00.108441 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:00.108790 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:00.109618 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:00.110681 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:00.111538 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:00.112012 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:00.112183 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:00.115513 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:00.115518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:00.115531 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:00.115559 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:00.117294 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:00.118034 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:00.118039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:00.118043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:00.118153 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:00.118549 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:00.118551 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:00.118552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:00.118932 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:00.119097 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:00.119478 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 19:03:00.119726 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":156,"threads_started":1}
I20260812 19:03:00.120829 21867 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:00.120932 21867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:00.121927 21867 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6aee5bbc4572941ee60137e2bdbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } }
I20260812 19:03:00.122290 21867 leader_election.cc:302] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdeb6aee5bbc4572941ee60137e2bdbc; no voters:
I20260812 19:03:00.122677 21867 leader_election.cc:288] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:00.122781 21872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:00.122988 21872 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [term 1 LEADER]: Becoming Leader. State: Replica: fdeb6aee5bbc4572941ee60137e2bdbc, State: Running, Role: LEADER
I20260812 19:03:00.123287 21872 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6aee5bbc4572941ee60137e2bdbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } }
I20260812 19:03:00.123968 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:00.124064 21867 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:00.124465 21872 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdeb6aee5bbc4572941ee60137e2bdbc. Latest consensus state: current_term: 1 leader_uuid: "fdeb6aee5bbc4572941ee60137e2bdbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6aee5bbc4572941ee60137e2bdbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } } }
I20260812 19:03:00.124609 21872 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:00.126871 21875 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdeb6aee5bbc4572941ee60137e2bdbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6aee5bbc4572941ee60137e2bdbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } } }
I20260812 19:03:00.126971 21875 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:00.127264 21877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:00.130393 21877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:00.135576 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:00.126701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55891 (local address 127.0.0.1:43339)
0812 19:03:00.126827 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:00.126833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:00.127089 (+ 256us) server_negotiation.cc:407] Connection header received
0812 19:03:00.127682 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:00.127685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:00.127732 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:00.127814 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:00.128576 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:00.129120 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:00.129976 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:00.130224 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:00.130334 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:00.131752 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:00.131755 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:00.131758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:00.131783 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:00.133452 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:00.134120 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:00.134124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:00.134126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:00.134218 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:00.134632 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:00.134635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:00.134637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:00.134854 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:00.135001 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:00.135323 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 19:03:00.135426 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 19:03:00.136672 21877 catalog_manager.cc:1347] Generated new cluster ID: 7bd998b71ba244188158ef4295fb3ff1
I20260812 19:03:00.136732 21877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:00.156235 21877 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:00.157168 21877 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:00.162081 21877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdeb6aee5bbc4572941ee60137e2bdbc: Generated new TSK 0
I20260812 19:03:00.162724 21877 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:00.167330 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 21779
W20260812 19:03:00.177541 21761 connection.cc:535] client connection to 127.0.0.1:43339 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43339): Connection reset by peer (error 104)
2026-08-12T19:03:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:00 dist-test-slave-6nmv krb5kdc[21906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:00 dist-test-slave-6nmv krb5kdc[21906](info): set up 2 sockets
Aug 12 19:03:00 dist-test-slave-6nmv krb5kdc[21906](info): commencing operation
krb5kdc: starting...
W20260812 19:03:01.292054 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:01 dist-test-slave-6nmv krb5kdc[21906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:03:01.419057 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37643
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:01.537082 21920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:01.537338 21920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:01.537382 21920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:01.541265 21920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:01.541323 21920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:01.541342 21920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:01.541357 21920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:01.541373 21920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:01.546556 21920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37643
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:01.547551 21920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:01.548533 21920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:01.557175 21925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:01.557256 21930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:01.557156 21920 server_base.cc:1034] running on GCE node
W20260812 19:03:01.557160 21927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:01.558003 21920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:01.559283 21920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:01.560474 21920 hybrid_clock.cc:648] HybridClock initialized: now 1786561381560444 us; error 40 us; skew 500 ppm
Aug 12 19:03:01 dist-test-slave-6nmv krb5kdc[21906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561381, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:01.563477 21920 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:01.564414 21920 webserver.cc:460] Webserver started at http://127.0.0.1:36453/ using document root <none> and password file <none>
I20260812 19:03:01.564971 21920 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:01.565081 21920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:01.565325 21920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:01.567394 21920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "0f60cd22ed1e4b4bb57d3d786f71101e"
format_stamp: "Formatted at 2026-08-12 19:03:01 on dist-test-slave-6nmv"
server_key: "f1af0fd996999760083cb59002a7f275"
server_key_iv: "a17c17b40240201327aaa7a18aac3258"
server_key_version: "encryptionkey@0"
I20260812 19:03:01.567962 21920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "0f60cd22ed1e4b4bb57d3d786f71101e"
format_stamp: "Formatted at 2026-08-12 19:03:01 on dist-test-slave-6nmv"
server_key: "f1af0fd996999760083cb59002a7f275"
server_key_iv: "a17c17b40240201327aaa7a18aac3258"
server_key_version: "encryptionkey@0"
I20260812 19:03:01.571697 21920 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 19:03:01.574129 21938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:01.575313 21920 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 19:03:01.575450 21920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "0f60cd22ed1e4b4bb57d3d786f71101e"
format_stamp: "Formatted at 2026-08-12 19:03:01 on dist-test-slave-6nmv"
server_key: "f1af0fd996999760083cb59002a7f275"
server_key_iv: "a17c17b40240201327aaa7a18aac3258"
server_key_version: "encryptionkey@0"
I20260812 19:03:01.575557 21920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:01.587960 21920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:01.590927 21920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:01.591126 21920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:01.598433 21920 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37643
I20260812 19:03:01.598457 21997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37643 every 8 connection(s)
I20260812 19:03:01.599377 21920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:01.602219 21998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:01.606113 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 21920
I20260812 19:03:01.606307 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:01.606617 30147 external_mini_cluster.cc:1428] Setting key db8525f3bcb3bd4a22169fba288dd85f
I20260812 19:03:01.608232 21998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e: Bootstrap starting.
I20260812 19:03:01.610526 21998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:01.611244 21998 log.cc:824] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:01.613029 21998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e: No bootstrap required, opened a new log
I20260812 19:03:01.615307 22002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:01.608012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42679 (local address 127.0.0.1:37643)
0812 19:03:01.608464 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:01.608480 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:01.608519 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:03:01.609138 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:01.609154 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:01.609494 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:01.609823 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:01.610275 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:01.611321 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:01.612219 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:01.612679 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:01.612860 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:01.613724 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:01.613729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:01.613740 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:01.614167 (+ 427us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:01.614216 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:01.614238 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:01.614372 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:01.614546 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 19:03:01.614678 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":134,"threads_started":1}
I20260812 19:03:01.616562 21998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f60cd22ed1e4b4bb57d3d786f71101e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37643 } }
I20260812 19:03:01.616808 21998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:01.616854 21998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f60cd22ed1e4b4bb57d3d786f71101e, State: Initialized, Role: FOLLOWER
I20260812 19:03:01.617352 21998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f60cd22ed1e4b4bb57d3d786f71101e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37643 } }
I20260812 19:03:01.617544 21998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:01.617628 21998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:01.617735 21998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:01.618247 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:01.618744 21998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f60cd22ed1e4b4bb57d3d786f71101e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37643 } }
I20260812 19:03:01.619064 21998 leader_election.cc:302] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f60cd22ed1e4b4bb57d3d786f71101e; no voters:
I20260812 19:03:01.619433 21998 leader_election.cc:288] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:01.619511 22005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:01.619711 22005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [term 1 LEADER]: Becoming Leader. State: Replica: 0f60cd22ed1e4b4bb57d3d786f71101e, State: Running, Role: LEADER
I20260812 19:03:01.620057 22005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f60cd22ed1e4b4bb57d3d786f71101e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37643 } }
I20260812 19:03:01.620684 21998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:01.621703 22011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f60cd22ed1e4b4bb57d3d786f71101e. Latest consensus state: current_term: 1 leader_uuid: "0f60cd22ed1e4b4bb57d3d786f71101e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f60cd22ed1e4b4bb57d3d786f71101e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37643 } } }
I20260812 19:03:01.621925 22011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:01.622285 22010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f60cd22ed1e4b4bb57d3d786f71101e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f60cd22ed1e4b4bb57d3d786f71101e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37643 } } }
I20260812 19:03:01.622416 22010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:01.625425 22002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:01.620641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50725 (local address 127.0.0.1:37643)
0812 19:03:01.620731 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:01.620736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:01.620940 (+ 204us) server_negotiation.cc:407] Connection header received
0812 19:03:01.621056 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:01.621059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:01.621099 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:01.621188 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:01.621639 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:01.622421 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:01.623436 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:01.623766 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:01.623890 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:01.624744 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:01.624747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:01.624750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:01.624762 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:01.624790 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:01.624803 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:01.624897 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:01.625108 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 19:03:01.625295 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 19:03:01.625808 22027 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:03:01.625934 22027 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:01.626045 22019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:01.626820 22019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:01.631515 22019 catalog_manager.cc:1347] Generated new cluster ID: d2fd74b0c9e143608acde2bfa5157c05
I20260812 19:03:01.631578 22019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:01.646903 22019 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:01.648164 22019 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:01.654425 22019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f60cd22ed1e4b4bb57d3d786f71101e: Generated new TSK 0
I20260812 19:03:01.655215 22019 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:01.659317 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 21920
W20260812 19:03:01.669750 21901 connection.cc:535] client connection to 127.0.0.1:37643 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37643): Connection reset by peer (error 104)
2026-08-12T19:03:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:01 dist-test-slave-6nmv krb5kdc[22033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:01 dist-test-slave-6nmv krb5kdc[22033](info): set up 2 sockets
Aug 12 19:03:01 dist-test-slave-6nmv krb5kdc[22033](info): commencing operation
krb5kdc: starting...
W20260812 19:03:02.867553 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:02 dist-test-slave-6nmv krb5kdc[22033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:03:02.990311 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35545
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:03.101045 22049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:03.101298 22049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:03.101339 22049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:03.105304 22049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:03.105358 22049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:03.105378 22049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:03.105394 22049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:03.105409 22049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:03.110536 22049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35545
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:03.111509 22049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:03.112491 22049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:03.120662 22057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:03.120687 22054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:03.120662 22055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:03.121129 22049 server_base.cc:1034] running on GCE node
I20260812 19:03:03.121693 22049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:03.122848 22049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:03.124006 22049 hybrid_clock.cc:648] HybridClock initialized: now 1786561383123996 us; error 28 us; skew 500 ppm
Aug 12 19:03:03 dist-test-slave-6nmv krb5kdc[22033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561383, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:03.126888 22049 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:03.127784 22049 webserver.cc:460] Webserver started at http://127.0.0.1:35169/ using document root <none> and password file <none>
I20260812 19:03:03.128319 22049 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:03.128412 22049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:03.128650 22049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:03.130678 22049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "dda1b9c5129b44668597e4d28fc1d89e"
format_stamp: "Formatted at 2026-08-12 19:03:03 on dist-test-slave-6nmv"
server_key: "30561822af08aad564880b8229745732"
server_key_iv: "b01236e072cf1d995583d5753ed23c9d"
server_key_version: "encryptionkey@0"
I20260812 19:03:03.131228 22049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "dda1b9c5129b44668597e4d28fc1d89e"
format_stamp: "Formatted at 2026-08-12 19:03:03 on dist-test-slave-6nmv"
server_key: "30561822af08aad564880b8229745732"
server_key_iv: "b01236e072cf1d995583d5753ed23c9d"
server_key_version: "encryptionkey@0"
I20260812 19:03:03.134800 22049 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:03:03.137048 22068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:03.138244 22049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:03:03.138381 22049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "dda1b9c5129b44668597e4d28fc1d89e"
format_stamp: "Formatted at 2026-08-12 19:03:03 on dist-test-slave-6nmv"
server_key: "30561822af08aad564880b8229745732"
server_key_iv: "b01236e072cf1d995583d5753ed23c9d"
server_key_version: "encryptionkey@0"
I20260812 19:03:03.138486 22049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:03.142908 22049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:03.145632 22049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:03.145891 22049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:03.152860 22049 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35545
I20260812 19:03:03.152889 22126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35545 every 8 connection(s)
I20260812 19:03:03.153743 22049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:03.156383 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 22049
I20260812 19:03:03.156490 22127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:03.156522 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:03.156817 30147 external_mini_cluster.cc:1428] Setting key 1a7c3208852280ff4ea221a8035e7d18
I20260812 19:03:03.162727 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:03.158279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58301 (local address 127.0.0.1:35545)
0812 19:03:03.158694 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:03.158713 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:03.158755 (+ 42us) server_negotiation.cc:407] Connection header received
0812 19:03:03.159893 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:03.159917 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:03.160280 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:03.160652 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:03.161349 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:03.161354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:03.161369 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:03.161381 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:03.161445 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:03.161471 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:03.161617 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:03.161686 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 19:03:03.161856 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":117,"threads_started":1}
I20260812 19:03:03.163744 22127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e: Bootstrap starting.
I20260812 19:03:03.165518 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:03.166414 22127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:03.167337 22127 log.cc:824] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:03.169931 22127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e: No bootstrap required, opened a new log
I20260812 19:03:03.170082 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:03.167892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40451 (local address 127.0.0.1:35545)
0812 19:03:03.168019 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:03.168025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:03.168426 (+ 401us) server_negotiation.cc:407] Connection header received
0812 19:03:03.168635 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:03.168638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:03.168697 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:03.168784 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:03.169635 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:03.169638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:03.169641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:03.169653 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:03.169688 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:03.169699 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:03.169797 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:03.169907 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 19:03:03.169948 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:03:03.173786 22127 raft_consensus.cc:348] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1b9c5129b44668597e4d28fc1d89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260812 19:03:03.174139 22127 raft_consensus.cc:374] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:03.174233 22127 raft_consensus.cc:729] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda1b9c5129b44668597e4d28fc1d89e, State: Initialized, Role: FOLLOWER
I20260812 19:03:03.174655 22127 consensus_queue.cc:260] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1b9c5129b44668597e4d28fc1d89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260812 19:03:03.174810 22127 raft_consensus.cc:388] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:03.174898 22127 raft_consensus.cc:482] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:03.174997 22127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:03.175940 22127 raft_consensus.cc:504] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1b9c5129b44668597e4d28fc1d89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260812 19:03:03.176283 22127 leader_election.cc:302] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dda1b9c5129b44668597e4d28fc1d89e; no voters:
I20260812 19:03:03.176610 22127 leader_election.cc:288] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:03.176729 22138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:03.176959 22138 raft_consensus.cc:686] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [term 1 LEADER]: Becoming Leader. State: Replica: dda1b9c5129b44668597e4d28fc1d89e, State: Running, Role: LEADER
I20260812 19:03:03.177294 22138 consensus_queue.cc:237] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1b9c5129b44668597e4d28fc1d89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260812 19:03:03.177906 22127 sys_catalog.cc:564] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:03.179138 22140 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dda1b9c5129b44668597e4d28fc1d89e. Latest consensus state: current_term: 1 leader_uuid: "dda1b9c5129b44668597e4d28fc1d89e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1b9c5129b44668597e4d28fc1d89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } } }
I20260812 19:03:03.179179 22139 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dda1b9c5129b44668597e4d28fc1d89e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dda1b9c5129b44668597e4d28fc1d89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } } }
I20260812 19:03:03.179261 22140 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:03.179287 22139 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:03.179669 22147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:03.182663 22147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:03.188119 22147 catalog_manager.cc:1347] Generated new cluster ID: ea81888b81e1411c89c7b214595cac47
I20260812 19:03:03.188184 22147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:03.212777 22147 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:03.213133 22147 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:03.221025 22147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dda1b9c5129b44668597e4d28fc1d89e: Generated new TSK 0
I20260812 19:03:03.221803 22147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:03.230789 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 22049
2026-08-12T19:03:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:03 dist-test-slave-6nmv krb5kdc[22163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:03 dist-test-slave-6nmv krb5kdc[22163](info): set up 2 sockets
Aug 12 19:03:03 dist-test-slave-6nmv krb5kdc[22163](info): commencing operation
krb5kdc: starting...
W20260812 19:03:04.352582 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:04 dist-test-slave-6nmv krb5kdc[22163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:03:04.474838 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43749
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:04.598569 22179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:04.598831 22179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:04.598873 22179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:04.605381 22179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:04.605459 22179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:04.605495 22179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:04.605535 22179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:04.605563 22179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:04.611356 22179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:04.612439 22179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:04.613466 22179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:04.621773 22188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:04.621963 22191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:04.621803 22189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:04.622434 22179 server_base.cc:1034] running on GCE node
I20260812 19:03:04.622910 22179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:04.623905 22179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:04.625065 22179 hybrid_clock.cc:648] HybridClock initialized: now 1786561384625048 us; error 28 us; skew 500 ppm
Aug 12 19:03:04 dist-test-slave-6nmv krb5kdc[22163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561384, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:04.627859 22179 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:04.628762 22179 webserver.cc:460] Webserver started at http://127.0.0.1:39677/ using document root <none> and password file <none>
I20260812 19:03:04.629320 22179 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:04.629380 22179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:04.629590 22179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:04.631652 22179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "bbe6c51868de447fb73e21261c251743"
format_stamp: "Formatted at 2026-08-12 19:03:04 on dist-test-slave-6nmv"
server_key: "d5a961d0b9b63858e8d5060d05ef9a6c"
server_key_iv: "73d91a09a228df51068ce4325e4202b8"
server_key_version: "encryptionkey@0"
I20260812 19:03:04.632211 22179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "bbe6c51868de447fb73e21261c251743"
format_stamp: "Formatted at 2026-08-12 19:03:04 on dist-test-slave-6nmv"
server_key: "d5a961d0b9b63858e8d5060d05ef9a6c"
server_key_iv: "73d91a09a228df51068ce4325e4202b8"
server_key_version: "encryptionkey@0"
I20260812 19:03:04.635859 22179 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 19:03:04.638149 22200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:04.639303 22179 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 19:03:04.639451 22179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "bbe6c51868de447fb73e21261c251743"
format_stamp: "Formatted at 2026-08-12 19:03:04 on dist-test-slave-6nmv"
server_key: "d5a961d0b9b63858e8d5060d05ef9a6c"
server_key_iv: "73d91a09a228df51068ce4325e4202b8"
server_key_version: "encryptionkey@0"
I20260812 19:03:04.639550 22179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:04.659336 22179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:04.662331 22179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:04.662534 22179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:04.669691 22179 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43749
I20260812 19:03:04.669720 22264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43749 every 8 connection(s)
I20260812 19:03:04.670603 22179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:04.671643 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 22179
I20260812 19:03:04.671818 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:04.672128 30147 external_mini_cluster.cc:1428] Setting key ff834bfa939c1272c2ff2c272fc5b046
I20260812 19:03:04.674041 22265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:04.679306 22265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743: Bootstrap starting.
Aug 12 19:03:04 dist-test-slave-6nmv krb5kdc[22163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 19:03:04.681509 22265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:04.682436 22265 log.cc:824] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:04.684234 22265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743: No bootstrap required, opened a new log
I20260812 19:03:04.685897 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:04.673573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52267 (local address 127.0.0.1:43749)
0812 19:03:04.674038 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:04.674053 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:04.674087 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:03:04.674756 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:04.674771 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:04.675092 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:04.675412 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:04.676207 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:04.677175 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:04.678122 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:04.678580 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:04.678771 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:04.681193 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:04.681198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:04.681209 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:04.681235 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:04.683151 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:04.683599 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:04.683605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:04.683611 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:04.683745 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:04.684035 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:04.684038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:04.684040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:04.684450 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:04.684656 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:04.684899 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 19:03:04.685183 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":166,"threads_started":1}
I20260812 19:03:04.688169 22265 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe6c51868de447fb73e21261c251743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43749 } }
I20260812 19:03:04.688442 22265 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:04.688510 22265 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe6c51868de447fb73e21261c251743, State: Initialized, Role: FOLLOWER
I20260812 19:03:04.688836 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:04.688966 22265 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe6c51868de447fb73e21261c251743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43749 } }
I20260812 19:03:04.689111 22265 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:04.689167 22265 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:04.689253 22265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:04.690508 22265 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe6c51868de447fb73e21261c251743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43749 } }
I20260812 19:03:04.690866 22265 leader_election.cc:302] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbe6c51868de447fb73e21261c251743; no voters:
I20260812 19:03:04.691255 22265 leader_election.cc:288] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:04.691408 22275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:04.691648 22275 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [term 1 LEADER]: Becoming Leader. State: Replica: bbe6c51868de447fb73e21261c251743, State: Running, Role: LEADER
I20260812 19:03:04.691967 22275 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe6c51868de447fb73e21261c251743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43749 } }
I20260812 19:03:04.692528 22265 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:04.693266 22275 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbe6c51868de447fb73e21261c251743. Latest consensus state: current_term: 1 leader_uuid: "bbe6c51868de447fb73e21261c251743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe6c51868de447fb73e21261c251743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43749 } } }
I20260812 19:03:04.693375 22275 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:04.693701 22278 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbe6c51868de447fb73e21261c251743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbe6c51868de447fb73e21261c251743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43749 } } }
I20260812 19:03:04.693796 22278 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:04.694237 22284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:04.697752 22284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:04.700580 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:04.691258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41389 (local address 127.0.0.1:43749)
0812 19:03:04.691374 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:04.691378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:04.691662 (+ 284us) server_negotiation.cc:407] Connection header received
0812 19:03:04.691815 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:04.691818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:04.691862 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:04.691910 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:04.692858 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:04.693647 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:04.694827 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:04.695183 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:04.695261 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:04.696849 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:04.696855 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:04.696859 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:04.696882 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:04.698730 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:04.699217 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:04.699221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:04.699223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:04.699311 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:04.699660 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:04.699664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:04.699665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:04.699884 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:04.700010 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:04.700262 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 19:03:04.700388 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:03:04.705978 22284 catalog_manager.cc:1347] Generated new cluster ID: a6272c8714104435a43c675ca9ce7e3c
I20260812 19:03:04.706087 22284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:04.721040 22284 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:04.722227 22284 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:04.740144 22284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbe6c51868de447fb73e21261c251743: Generated new TSK 0
I20260812 19:03:04.740871 22284 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:04.755125 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 22179
2026-08-12T19:03:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:04 dist-test-slave-6nmv krb5kdc[22299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:04 dist-test-slave-6nmv krb5kdc[22299](info): set up 2 sockets
Aug 12 19:03:04 dist-test-slave-6nmv krb5kdc[22299](info): commencing operation
krb5kdc: starting...
W20260812 19:03:05.876098 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:05 dist-test-slave-6nmv krb5kdc[22299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:03:05.998833 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38953
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:06.106913 22316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:06.107205 22316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:06.107249 22316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:06.111308 22316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:06.111371 22316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:06.111390 22316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:06.111406 22316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:06.111423 22316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:06.116824 22316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:06.117856 22316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:06.118819 22316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:06.127393 22323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:06.127427 22316 server_base.cc:1034] running on GCE node
W20260812 19:03:06.127553 22324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:06.127642 22326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:06.128271 22316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:06.129345 22316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:06.130538 22316 hybrid_clock.cc:648] HybridClock initialized: now 1786561386130521 us; error 30 us; skew 500 ppm
Aug 12 19:03:06 dist-test-slave-6nmv krb5kdc[22299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:06.133430 22316 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:06.134333 22316 webserver.cc:460] Webserver started at http://127.0.0.1:46711/ using document root <none> and password file <none>
I20260812 19:03:06.134845 22316 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:06.134902 22316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:06.135066 22316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:06.136992 22316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "c1e53f19c202482b9d15be130e5d9e03"
format_stamp: "Formatted at 2026-08-12 19:03:06 on dist-test-slave-6nmv"
server_key: "64b4e6d1fbfc302f5628459000aceb60"
server_key_iv: "afd5e7868ab84c9404d27829528012eb"
server_key_version: "encryptionkey@0"
I20260812 19:03:06.137480 22316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "c1e53f19c202482b9d15be130e5d9e03"
format_stamp: "Formatted at 2026-08-12 19:03:06 on dist-test-slave-6nmv"
server_key: "64b4e6d1fbfc302f5628459000aceb60"
server_key_iv: "afd5e7868ab84c9404d27829528012eb"
server_key_version: "encryptionkey@0"
I20260812 19:03:06.141129 22316 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 19:03:06.143498 22334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:06.144649 22316 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:03:06.144790 22316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "c1e53f19c202482b9d15be130e5d9e03"
format_stamp: "Formatted at 2026-08-12 19:03:06 on dist-test-slave-6nmv"
server_key: "64b4e6d1fbfc302f5628459000aceb60"
server_key_iv: "afd5e7868ab84c9404d27829528012eb"
server_key_version: "encryptionkey@0"
I20260812 19:03:06.144919 22316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:06.161530 22316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:06.165076 22316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:06.165290 22316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:06.172744 22316 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38953
I20260812 19:03:06.172783 22407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38953 every 8 connection(s)
I20260812 19:03:06.173671 22316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:06.174779 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 22316
I20260812 19:03:06.174919 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:06.175204 30147 external_mini_cluster.cc:1428] Setting key 4e9eccfbd1d61a057c026fba2a86c14a
I20260812 19:03:06.176792 22408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:06.182178 22408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03: Bootstrap starting.
I20260812 19:03:06.183729 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:06.176544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58281 (local address 127.0.0.1:38953)
0812 19:03:06.177015 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:06.177030 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:06.177065 (+ 35us) server_negotiation.cc:407] Connection header received
0812 19:03:06.177663 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:06.177677 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:06.178108 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:06.178455 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:06.178850 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:06.179889 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:06.180769 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:06.181215 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:06.181368 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:06.182166 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:06.182170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:06.182181 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:06.182591 (+ 410us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:06.182645 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:06.182668 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:06.182772 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:06.182883 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 19:03:06.183083 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":137,"threads_started":1}
I20260812 19:03:06.184448 22408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:06.185180 22408 log.cc:824] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:06.186525 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:06.187227 22408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03: No bootstrap required, opened a new log
I20260812 19:03:06.190702 22408 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e53f19c202482b9d15be130e5d9e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38953 } }
I20260812 19:03:06.190955 22408 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:06.191040 22408 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e53f19c202482b9d15be130e5d9e03, State: Initialized, Role: FOLLOWER
I20260812 19:03:06.191440 22408 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e53f19c202482b9d15be130e5d9e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38953 } }
I20260812 19:03:06.191601 22408 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:06.191684 22408 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:06.191773 22408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:06.192617 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:06.188585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51591 (local address 127.0.0.1:38953)
0812 19:03:06.188713 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:06.188719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:06.188931 (+ 212us) server_negotiation.cc:407] Connection header received
0812 19:03:06.189097 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:06.189099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:06.189135 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:06.189213 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:06.189483 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:06.190031 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:06.190845 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:06.191180 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:06.191290 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:06.192107 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:06.192110 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:06.192113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:06.192142 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:06.192175 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:06.192185 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:06.192260 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:06.192410 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 19:03:06.192460 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 19:03:06.192667 22408 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e53f19c202482b9d15be130e5d9e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38953 } }
I20260812 19:03:06.193058 22408 leader_election.cc:302] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1e53f19c202482b9d15be130e5d9e03; no voters:
I20260812 19:03:06.193375 22408 leader_election.cc:288] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:06.193496 22419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:06.193776 22419 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [term 1 LEADER]: Becoming Leader. State: Replica: c1e53f19c202482b9d15be130e5d9e03, State: Running, Role: LEADER
I20260812 19:03:06.194145 22419 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e53f19c202482b9d15be130e5d9e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38953 } }
I20260812 19:03:06.194561 22408 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:06.195701 22421 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1e53f19c202482b9d15be130e5d9e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e53f19c202482b9d15be130e5d9e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38953 } } }
I20260812 19:03:06.195832 22421 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:06.196242 22420 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e53f19c202482b9d15be130e5d9e03. Latest consensus state: current_term: 1 leader_uuid: "c1e53f19c202482b9d15be130e5d9e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e53f19c202482b9d15be130e5d9e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38953 } } }
I20260812 19:03:06.196383 22420 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03 [sys.catalog]: This master's current role is: LEADER
W20260812 19:03:06.199182 22434 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:03:06.199245 22434 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:06.199321 22428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:06.200076 22428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:06.206272 22428 catalog_manager.cc:1347] Generated new cluster ID: 560005373a0e4b61bc13975066494fcb
I20260812 19:03:06.206343 22428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:06.216195 22428 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:06.216957 22428 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:06.221923 22428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1e53f19c202482b9d15be130e5d9e03: Generated new TSK 0
I20260812 19:03:06.222527 22428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:06.226718 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 22316
2026-08-12T19:03:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1473 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:06 dist-test-slave-6nmv krb5kdc[22441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:06 dist-test-slave-6nmv krb5kdc[22441](info): set up 2 sockets
Aug 12 19:03:06 dist-test-slave-6nmv krb5kdc[22441](info): commencing operation
krb5kdc: starting...
W20260812 19:03:07.355783 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:07 dist-test-slave-6nmv krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:03:07.481273 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34687
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:07.592590 22456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:07.592934 22456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:07.593026 22456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:07.597119 22456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:07.597215 22456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:07.597273 22456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:07.597342 22456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:07.597394 22456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:07.602586 22456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:07.603657 22456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:07.604668 22456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:07.613389 22468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:07.613358 22465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:07.613462 22464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:07.613487 22456 server_base.cc:1034] running on GCE node
I20260812 19:03:07.614252 22456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:07.615329 22456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:07.616518 22456 hybrid_clock.cc:648] HybridClock initialized: now 1786561387616504 us; error 35 us; skew 500 ppm
Aug 12 19:03:07 dist-test-slave-6nmv krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561387, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:07.619253 22456 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:07.620180 22456 webserver.cc:460] Webserver started at http://127.0.0.1:36785/ using document root <none> and password file <none>
I20260812 19:03:07.620733 22456 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:07.620823 22456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:07.621068 22456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:07.623183 22456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "d4c890c6e31d4baba1510475779635cb"
format_stamp: "Formatted at 2026-08-12 19:03:07 on dist-test-slave-6nmv"
server_key: "4bb7a9c97bf69867230fc1b437b2611a"
server_key_iv: "e9a301b1b7eb0086244b9124e06ad857"
server_key_version: "encryptionkey@0"
I20260812 19:03:07.623821 22456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "d4c890c6e31d4baba1510475779635cb"
format_stamp: "Formatted at 2026-08-12 19:03:07 on dist-test-slave-6nmv"
server_key: "4bb7a9c97bf69867230fc1b437b2611a"
server_key_iv: "e9a301b1b7eb0086244b9124e06ad857"
server_key_version: "encryptionkey@0"
I20260812 19:03:07.627527 22456 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 19:03:07.629791 22476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:07.630991 22456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:07.631150 22456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "d4c890c6e31d4baba1510475779635cb"
format_stamp: "Formatted at 2026-08-12 19:03:07 on dist-test-slave-6nmv"
server_key: "4bb7a9c97bf69867230fc1b437b2611a"
server_key_iv: "e9a301b1b7eb0086244b9124e06ad857"
server_key_version: "encryptionkey@0"
I20260812 19:03:07.631268 22456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:07.635820 22456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:07.638777 22456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:07.638918 22456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:07.645546 22456 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34687
I20260812 19:03:07.645558 22539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34687 every 8 connection(s)
I20260812 19:03:07.646517 22456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:07.647159 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 22456
I20260812 19:03:07.647310 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:07.647598 30147 external_mini_cluster.cc:1428] Setting key 619d83e351dcb24d0925eb9e1d984b30
I20260812 19:03:07.649610 22540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:07.652632 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:07.648881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57051 (local address 127.0.0.1:34687)
0812 19:03:07.649287 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:07.649301 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:07.649335 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:03:07.650094 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:07.650111 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:07.650370 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:07.650640 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:07.651457 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:07.651461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:07.651472 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:07.651482 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:07.651529 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:07.651547 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:07.651679 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:07.651787 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 19:03:07.651944 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":132,"threads_started":1}
I20260812 19:03:07.655504 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:07.655959 22540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb: Bootstrap starting.
I20260812 19:03:07.658613 22540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:07.659340 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:07.657727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59881 (local address 127.0.0.1:34687)
0812 19:03:07.657873 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:07.657877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:07.658102 (+ 225us) server_negotiation.cc:407] Connection header received
0812 19:03:07.658187 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:07.658190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:07.658222 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:07.658295 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:07.658952 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:07.658955 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:07.658958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:07.658969 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:07.659004 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:07.659018 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:07.659104 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:07.659192 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 19:03:07.659225 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:03:07.659516 22540 log.cc:824] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:07.661309 22540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb: No bootstrap required, opened a new log
I20260812 19:03:07.664656 22540 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c890c6e31d4baba1510475779635cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } }
I20260812 19:03:07.664880 22540 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:07.664960 22540 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c890c6e31d4baba1510475779635cb, State: Initialized, Role: FOLLOWER
I20260812 19:03:07.665416 22540 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c890c6e31d4baba1510475779635cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } }
I20260812 19:03:07.665556 22540 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:07.665629 22540 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:07.665727 22540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:07.666781 22540 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c890c6e31d4baba1510475779635cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } }
I20260812 19:03:07.667091 22540 leader_election.cc:302] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4c890c6e31d4baba1510475779635cb; no voters:
I20260812 19:03:07.667356 22540 leader_election.cc:288] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:07.667492 22555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:07.667738 22555 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [term 1 LEADER]: Becoming Leader. State: Replica: d4c890c6e31d4baba1510475779635cb, State: Running, Role: LEADER
I20260812 19:03:07.668063 22555 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c890c6e31d4baba1510475779635cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } }
I20260812 19:03:07.668480 22540 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:07.669579 22557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4c890c6e31d4baba1510475779635cb. Latest consensus state: current_term: 1 leader_uuid: "d4c890c6e31d4baba1510475779635cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c890c6e31d4baba1510475779635cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } } }
I20260812 19:03:07.669718 22557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:07.669947 22556 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4c890c6e31d4baba1510475779635cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c890c6e31d4baba1510475779635cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34687 } } }
I20260812 19:03:07.670063 22556 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:07.670970 22566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:07.673919 22566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:07.678824 22566 catalog_manager.cc:1347] Generated new cluster ID: d2f973e60350488e8e9d010664709f90
I20260812 19:03:07.678887 22566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:07.689488 22566 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:07.689781 22566 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:07.696624 22566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4c890c6e31d4baba1510475779635cb: Generated new TSK 0
I20260812 19:03:07.697201 22566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:07.703044 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 22456
2026-08-12T19:03:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:07 dist-test-slave-6nmv krb5kdc[22579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:07 dist-test-slave-6nmv krb5kdc[22579](info): set up 2 sockets
Aug 12 19:03:07 dist-test-slave-6nmv krb5kdc[22579](info): commencing operation
krb5kdc: starting...
W20260812 19:03:08.807766 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:08 dist-test-slave-6nmv krb5kdc[22579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:03:08.935295 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:09.048110 22592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:09.048370 22592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:09.048414 22592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:09.052429 22592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:09.052495 22592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:09.052513 22592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:09.052528 22592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:09.052543 22592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:09.057667 22592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:09.058766 22592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:09.059798 22592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:03:09.068222 22592 server_base.cc:1034] running on GCE node
W20260812 19:03:09.068186 22598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:09.068198 22597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:09.068186 22600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:09.069034 22592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:09.070199 22592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:09.071389 22592 hybrid_clock.cc:648] HybridClock initialized: now 1786561389071381 us; error 59 us; skew 500 ppm
Aug 12 19:03:09 dist-test-slave-6nmv krb5kdc[22579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:09.074173 22592 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:09.074980 22592 webserver.cc:460] Webserver started at http://127.0.0.1:32937/ using document root <none> and password file <none>
I20260812 19:03:09.075498 22592 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:09.075555 22592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:09.075712 22592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:09.077600 22592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "56ac691297244b3d85594f36ade1531c"
format_stamp: "Formatted at 2026-08-12 19:03:09 on dist-test-slave-6nmv"
server_key: "d2eac7e7529e9e6267e48821735073f6"
server_key_iv: "1c9e7616d3542eefc9848d0d7077e1bd"
server_key_version: "encryptionkey@0"
I20260812 19:03:09.078190 22592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "56ac691297244b3d85594f36ade1531c"
format_stamp: "Formatted at 2026-08-12 19:03:09 on dist-test-slave-6nmv"
server_key: "d2eac7e7529e9e6267e48821735073f6"
server_key_iv: "1c9e7616d3542eefc9848d0d7077e1bd"
server_key_version: "encryptionkey@0"
I20260812 19:03:09.081663 22592 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:09.084024 22610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:09.085183 22592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:09.085282 22592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "56ac691297244b3d85594f36ade1531c"
format_stamp: "Formatted at 2026-08-12 19:03:09 on dist-test-slave-6nmv"
server_key: "d2eac7e7529e9e6267e48821735073f6"
server_key_iv: "1c9e7616d3542eefc9848d0d7077e1bd"
server_key_version: "encryptionkey@0"
I20260812 19:03:09.085415 22592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:09.102560 22592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:09.105700 22592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:09.105937 22592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:09.113188 22592 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42205
I20260812 19:03:09.113263 22668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42205 every 8 connection(s)
I20260812 19:03:09.114205 22592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:09.117100 22669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:09.121941 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 22592
I20260812 19:03:09.122098 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:09.122409 30147 external_mini_cluster.cc:1428] Setting key f8c0edcd78b4b4484dcea20b597a59dc
I20260812 19:03:09.122762 22669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c: Bootstrap starting.
I20260812 19:03:09.125380 22669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:09.126204 22669 log.cc:824] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:09.128093 22669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c: No bootstrap required, opened a new log
Aug 12 19:03:09 dist-test-slave-6nmv krb5kdc[22579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 19:03:09.132532 22669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ac691297244b3d85594f36ade1531c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } }
I20260812 19:03:09.132814 22669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:09.132886 22669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56ac691297244b3d85594f36ade1531c, State: Initialized, Role: FOLLOWER
I20260812 19:03:09.133405 22669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ac691297244b3d85594f36ade1531c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } }
I20260812 19:03:09.133581 22669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:09.133658 22669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:09.133757 22669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:09.135049 22669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ac691297244b3d85594f36ade1531c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } }
I20260812 19:03:09.135421 22669 leader_election.cc:302] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56ac691297244b3d85594f36ade1531c; no voters:
I20260812 19:03:09.135779 22669 leader_election.cc:288] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:09.135910 22677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:09.136207 22677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [term 1 LEADER]: Becoming Leader. State: Replica: 56ac691297244b3d85594f36ade1531c, State: Running, Role: LEADER
I20260812 19:03:09.136642 22677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ac691297244b3d85594f36ade1531c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } }
I20260812 19:03:09.137040 22669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:09.137969 22677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56ac691297244b3d85594f36ade1531c. Latest consensus state: current_term: 1 leader_uuid: "56ac691297244b3d85594f36ade1531c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ac691297244b3d85594f36ade1531c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } } }
I20260812 19:03:09.138155 22677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:09.138408 22678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56ac691297244b3d85594f36ade1531c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56ac691297244b3d85594f36ade1531c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42205 } } }
I20260812 19:03:09.138522 22678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:09.138545 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:09.123807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:49391 (local address 127.0.0.1:42205)
0812 19:03:09.124238 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:09.124254 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:09.124290 (+ 36us) server_negotiation.cc:407] Connection header received
0812 19:03:09.124913 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:09.124928 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:09.125259 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:09.125617 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:09.126467 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:09.127434 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:09.128307 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:09.128978 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:09.129144 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:09.132134 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:09.132140 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:09.132155 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:09.132185 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:09.134327 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:09.134916 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:09.134925 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:09.134933 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:09.135084 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:09.135465 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:09.135469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:09.135471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:09.136045 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:09.136252 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:09.137231 (+ 979us) server_negotiation.cc:299] Negotiation successful
0812 19:03:09.137470 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":141,"threads_started":1}
I20260812 19:03:09.138617 22686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:09.141626 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:09.142688 22686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:09.150216 22686 catalog_manager.cc:1347] Generated new cluster ID: f582461243dd4f0da0c8e7e05673d841
I20260812 19:03:09.150328 22686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:09.152709 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:09.143870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:37779 (local address 127.0.0.1:42205)
0812 19:03:09.143993 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:09.143997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:09.144141 (+ 144us) server_negotiation.cc:407] Connection header received
0812 19:03:09.144212 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:09.144215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:09.144265 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:09.144354 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:09.145043 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:09.145592 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:09.146606 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:09.146976 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:09.147099 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:09.148668 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:09.148673 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:09.148675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:09.148704 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:09.150620 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:09.151209 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:09.151213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:09.151215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:09.151314 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:09.151723 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:09.151727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:09.151729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:09.151979 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:09.152134 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:09.152420 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 19:03:09.152535 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 19:03:09.166965 22686 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:09.168255 22686 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:09.172394 22686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56ac691297244b3d85594f36ade1531c: Generated new TSK 0
I20260812 19:03:09.173161 22686 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:09.180624 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 22592
W20260812 19:03:09.192487 22576 connection.cc:535] client connection to 127.0.0.1:42205 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42205): Connection reset by peer (error 104)
2026-08-12T19:03:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:09 dist-test-slave-6nmv krb5kdc[22707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:09 dist-test-slave-6nmv krb5kdc[22707](info): set up 2 sockets
Aug 12 19:03:09 dist-test-slave-6nmv krb5kdc[22707](info): commencing operation
krb5kdc: starting...
W20260812 19:03:10.302240 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:10 dist-test-slave-6nmv krb5kdc[22707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:03:10.424427 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:10.532972 22725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:10.533274 22725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:10.533355 22725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:10.537261 22725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:10.537350 22725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:10.537388 22725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:10.537442 22725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:10.537480 22725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:10.542623 22725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:10.543586 22725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:10.544533 22725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:10.552708 22733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:10.552735 22736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:10.552871 22725 server_base.cc:1034] running on GCE node
W20260812 19:03:10.552735 22731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:10.553504 22725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:10.554634 22725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:10.555816 22725 hybrid_clock.cc:648] HybridClock initialized: now 1786561390555791 us; error 38 us; skew 500 ppm
Aug 12 19:03:10 dist-test-slave-6nmv krb5kdc[22707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561390, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:10.558512 22725 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:10.559335 22725 webserver.cc:460] Webserver started at http://127.0.0.1:37209/ using document root <none> and password file <none>
I20260812 19:03:10.559896 22725 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:10.559984 22725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:10.560215 22725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:10.562191 22725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "8cea65c406ad4b8a96cae09ba8bfa963"
format_stamp: "Formatted at 2026-08-12 19:03:10 on dist-test-slave-6nmv"
server_key: "7d1a1b529367214b6b35a3c7057b442b"
server_key_iv: "3b745064835c94991f384ea73ff12a4a"
server_key_version: "encryptionkey@0"
I20260812 19:03:10.562731 22725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "8cea65c406ad4b8a96cae09ba8bfa963"
format_stamp: "Formatted at 2026-08-12 19:03:10 on dist-test-slave-6nmv"
server_key: "7d1a1b529367214b6b35a3c7057b442b"
server_key_iv: "3b745064835c94991f384ea73ff12a4a"
server_key_version: "encryptionkey@0"
I20260812 19:03:10.566267 22725 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:10.568578 22743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:10.569680 22725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:10.569864 22725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "8cea65c406ad4b8a96cae09ba8bfa963"
format_stamp: "Formatted at 2026-08-12 19:03:10 on dist-test-slave-6nmv"
server_key: "7d1a1b529367214b6b35a3c7057b442b"
server_key_iv: "3b745064835c94991f384ea73ff12a4a"
server_key_version: "encryptionkey@0"
I20260812 19:03:10.569970 22725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:10.598770 22725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:10.603093 22725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:10.603350 22725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:10.611886 22725 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42761
I20260812 19:03:10.611909 22807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42761 every 8 connection(s)
I20260812 19:03:10.612861 22725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:10.615576 22808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:10.620674 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 22725
I20260812 19:03:10.620818 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:10.621110 30147 external_mini_cluster.cc:1428] Setting key 57303178b94d0b61411f89ed2f516e01
I20260812 19:03:10.621179 22808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963: Bootstrap starting.
I20260812 19:03:10.623677 22808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:10.624364 22808 log.cc:824] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:10.626250 22808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963: No bootstrap required, opened a new log
I20260812 19:03:10.629755 22808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cea65c406ad4b8a96cae09ba8bfa963" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260812 19:03:10.630055 22808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:10.630153 22808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cea65c406ad4b8a96cae09ba8bfa963, State: Initialized, Role: FOLLOWER
I20260812 19:03:10.630622 22808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cea65c406ad4b8a96cae09ba8bfa963" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260812 19:03:10.630784 22808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:10.630851 22808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:10.630945 22808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:10.631817 22808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cea65c406ad4b8a96cae09ba8bfa963" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260812 19:03:10.632155 22808 leader_election.cc:302] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cea65c406ad4b8a96cae09ba8bfa963; no voters:
I20260812 19:03:10.632455 22808 leader_election.cc:288] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:10.632586 22813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:10.632840 22813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [term 1 LEADER]: Becoming Leader. State: Replica: 8cea65c406ad4b8a96cae09ba8bfa963, State: Running, Role: LEADER
I20260812 19:03:10.633495 22813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cea65c406ad4b8a96cae09ba8bfa963" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260812 19:03:10.633740 22808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:10.635202 22814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cea65c406ad4b8a96cae09ba8bfa963" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cea65c406ad4b8a96cae09ba8bfa963" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } } }
I20260812 19:03:10.635317 22814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:10.635433 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:10.622486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:52913 (local address 127.0.0.1:42761)
0812 19:03:10.622961 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:10.622978 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:10.623017 (+ 39us) server_negotiation.cc:407] Connection header received
0812 19:03:10.623704 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:10.623721 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:10.624156 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:10.624508 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:10.624961 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:10.626064 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:10.627087 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:10.627591 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:10.627775 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:10.633262 (+ 5487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:10.633268 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:10.633284 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:10.633798 (+ 514us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:10.633892 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:10.633923 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:10.634069 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:10.634296 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 19:03:10.634422 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":134,"threads_started":1}
I20260812 19:03:10.635865 22815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cea65c406ad4b8a96cae09ba8bfa963. Latest consensus state: current_term: 1 leader_uuid: "8cea65c406ad4b8a96cae09ba8bfa963" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cea65c406ad4b8a96cae09ba8bfa963" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } } }
I20260812 19:03:10.636482 22815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:10.636843 22826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:10.638815 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:10.639815 22826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:10.645716 22826 catalog_manager.cc:1347] Generated new cluster ID: d6f0b62889d24030ba2d5952a7dae609
I20260812 19:03:10.645776 22826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:10.650653 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:10.641044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:46905 (local address 127.0.0.1:42761)
0812 19:03:10.641156 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:10.641160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:10.641731 (+ 571us) server_negotiation.cc:407] Connection header received
0812 19:03:10.641964 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:10.641967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:10.642016 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:10.642074 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:10.642552 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:10.643073 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:10.643954 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:10.645066 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:10.645181 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:10.650194 (+ 5013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:10.650198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:10.650201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:10.650216 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:10.650246 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:10.650259 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:10.650356 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:10.650472 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 19:03:10.650507 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 19:03:10.656749 22826 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:10.657650 22826 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:10.662361 22826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cea65c406ad4b8a96cae09ba8bfa963: Generated new TSK 0
I20260812 19:03:10.663007 22826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:10.667126 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 22725
W20260812 19:03:10.680137 22705 connection.cc:535] client connection to 127.0.0.1:42761 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42761): Connection reset by peer (error 104)
2026-08-12T19:03:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:10 dist-test-slave-6nmv krb5kdc[22844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:10 dist-test-slave-6nmv krb5kdc[22844](info): set up 2 sockets
Aug 12 19:03:10 dist-test-slave-6nmv krb5kdc[22844](info): commencing operation
krb5kdc: starting...
W20260812 19:03:11.795676 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:11 dist-test-slave-6nmv krb5kdc[22844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:03:11.924228 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42093
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:12.037154 22862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:12.037467 22862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:12.037576 22862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:12.041674 22862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:12.041762 22862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:12.041800 22862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:12.041857 22862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:12.041891 22862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:12.047055 22862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:12.048131 22862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:12.049165 22862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:12.057989 22872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:12.057992 22867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:12.057991 22868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:12.058246 22862 server_base.cc:1034] running on GCE node
I20260812 19:03:12.058977 22862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:12.060145 22862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:12.061414 22862 hybrid_clock.cc:648] HybridClock initialized: now 1786561392061403 us; error 33 us; skew 500 ppm
Aug 12 19:03:12 dist-test-slave-6nmv krb5kdc[22844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561392, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:12.064466 22862 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:12.065361 22862 webserver.cc:460] Webserver started at http://127.0.0.1:45795/ using document root <none> and password file <none>
I20260812 19:03:12.065990 22862 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:12.066105 22862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:12.066344 22862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:12.068385 22862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "e06a141f33f34a7cb50427c294e51294"
format_stamp: "Formatted at 2026-08-12 19:03:12 on dist-test-slave-6nmv"
server_key: "9cefb189e5c74440d3462dc57ef0f233"
server_key_iv: "2471da9e61c4c1f98716c1fc64bdde30"
server_key_version: "encryptionkey@0"
I20260812 19:03:12.068953 22862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "e06a141f33f34a7cb50427c294e51294"
format_stamp: "Formatted at 2026-08-12 19:03:12 on dist-test-slave-6nmv"
server_key: "9cefb189e5c74440d3462dc57ef0f233"
server_key_iv: "2471da9e61c4c1f98716c1fc64bdde30"
server_key_version: "encryptionkey@0"
I20260812 19:03:12.073295 22862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260812 19:03:12.075831 22882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:12.077119 22862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:12.077265 22862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "e06a141f33f34a7cb50427c294e51294"
format_stamp: "Formatted at 2026-08-12 19:03:12 on dist-test-slave-6nmv"
server_key: "9cefb189e5c74440d3462dc57ef0f233"
server_key_iv: "2471da9e61c4c1f98716c1fc64bdde30"
server_key_version: "encryptionkey@0"
I20260812 19:03:12.077378 22862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:12.082327 22862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:12.085508 22862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:12.085731 22862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:12.093371 22862 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42093
I20260812 19:03:12.093380 22955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42093 every 8 connection(s)
I20260812 19:03:12.094432 22862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:12.097270 22956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:12.100981 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 22862
I20260812 19:03:12.101162 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:12.101456 30147 external_mini_cluster.cc:1428] Setting key b6c59ba3cfed6e6af96c07ef54dad819
I20260812 19:03:12.103605 22956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294: Bootstrap starting.
I20260812 19:03:12.106405 22956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:12.107012 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:12.102889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:49277 (local address 127.0.0.1:42093)
0812 19:03:12.103328 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:12.103344 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:12.103382 (+ 38us) server_negotiation.cc:407] Connection header received
0812 19:03:12.104073 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:12.104092 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:12.104388 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:12.104685 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:12.105635 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:12.105640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:12.105652 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:12.105663 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:12.105723 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:12.105743 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:12.105900 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:12.106016 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 19:03:12.106211 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":144,"threads_started":1}
I20260812 19:03:12.107252 22956 log.cc:824] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:12.109544 22956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294: No bootstrap required, opened a new log
I20260812 19:03:12.110136 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:12.114313 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:12.112507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:57805 (local address 127.0.0.1:42093)
0812 19:03:12.112640 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:12.112644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:12.112799 (+ 155us) server_negotiation.cc:407] Connection header received
0812 19:03:12.112958 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:12.112961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:12.112994 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:12.113067 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:12.113868 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:12.113875 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:12.113877 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:12.113893 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:12.113920 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:12.113931 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:12.114000 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:12.114125 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 19:03:12.114159 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 19:03:12.114475 22956 raft_consensus.cc:348] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a141f33f34a7cb50427c294e51294" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42093 } }
I20260812 19:03:12.114773 22956 raft_consensus.cc:374] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:12.114835 22956 raft_consensus.cc:729] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06a141f33f34a7cb50427c294e51294, State: Initialized, Role: FOLLOWER
I20260812 19:03:12.115288 22956 consensus_queue.cc:260] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a141f33f34a7cb50427c294e51294" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42093 } }
I20260812 19:03:12.115447 22956 raft_consensus.cc:388] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:12.115527 22956 raft_consensus.cc:482] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:12.115625 22956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:12.116559 22956 raft_consensus.cc:504] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a141f33f34a7cb50427c294e51294" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42093 } }
I20260812 19:03:12.116899 22956 leader_election.cc:302] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e06a141f33f34a7cb50427c294e51294; no voters:
I20260812 19:03:12.117220 22956 leader_election.cc:288] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:12.117295 22970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:12.117558 22970 raft_consensus.cc:686] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [term 1 LEADER]: Becoming Leader. State: Replica: e06a141f33f34a7cb50427c294e51294, State: Running, Role: LEADER
I20260812 19:03:12.118018 22970 consensus_queue.cc:237] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a141f33f34a7cb50427c294e51294" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42093 } }
I20260812 19:03:12.118520 22956 sys_catalog.cc:564] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:12.119766 22971 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e06a141f33f34a7cb50427c294e51294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a141f33f34a7cb50427c294e51294" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42093 } } }
I20260812 19:03:12.119812 22974 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e06a141f33f34a7cb50427c294e51294. Latest consensus state: current_term: 1 leader_uuid: "e06a141f33f34a7cb50427c294e51294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e06a141f33f34a7cb50427c294e51294" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42093 } } }
I20260812 19:03:12.119908 22971 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:12.119908 22974 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:12.120352 22981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:12.123463 22981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:12.129101 22981 catalog_manager.cc:1347] Generated new cluster ID: 8e7787d6fdce4d4caab34e4a3148cd1a
I20260812 19:03:12.129170 22981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:12.139166 22981 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:12.139499 22981 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:12.150836 22981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e06a141f33f34a7cb50427c294e51294: Generated new TSK 0
I20260812 19:03:12.151693 22981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:12.160373 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 22862
2026-08-12T19:03:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:12 dist-test-slave-6nmv krb5kdc[22995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:12 dist-test-slave-6nmv krb5kdc[22995](info): set up 2 sockets
Aug 12 19:03:12 dist-test-slave-6nmv krb5kdc[22995](info): commencing operation
krb5kdc: starting...
W20260812 19:03:13.281170 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:13 dist-test-slave-6nmv krb5kdc[22995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:03:13.410815 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34207
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:13.527328 23012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:13.527621 23012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:13.527668 23012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:13.531688 23012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:13.531764 23012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:13.531787 23012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:13.531807 23012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:13.531826 23012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:13.537261 23012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34207
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:13.538341 23012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:13.539330 23012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:13.548012 23019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:13.547959 23020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:13.547979 23023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:13.548229 23012 server_base.cc:1034] running on GCE node
I20260812 19:03:13.548838 23012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:13.550047 23012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:13.551245 23012 hybrid_clock.cc:648] HybridClock initialized: now 1786561393551225 us; error 31 us; skew 500 ppm
Aug 12 19:03:13 dist-test-slave-6nmv krb5kdc[22995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:13.554198 23012 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:13.555033 23012 webserver.cc:460] Webserver started at http://127.0.0.1:40161/ using document root <none> and password file <none>
I20260812 19:03:13.555543 23012 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:13.555601 23012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:13.555801 23012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:13.557775 23012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "e570e4d8f1734a13b84bbfd395e5a92e"
format_stamp: "Formatted at 2026-08-12 19:03:13 on dist-test-slave-6nmv"
server_key: "92a1476998d8c88dcf28023c69e7222a"
server_key_iv: "4899b1c9a53c8c0b66b2054ec3c84c2d"
server_key_version: "encryptionkey@0"
I20260812 19:03:13.558398 23012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "e570e4d8f1734a13b84bbfd395e5a92e"
format_stamp: "Formatted at 2026-08-12 19:03:13 on dist-test-slave-6nmv"
server_key: "92a1476998d8c88dcf28023c69e7222a"
server_key_iv: "4899b1c9a53c8c0b66b2054ec3c84c2d"
server_key_version: "encryptionkey@0"
I20260812 19:03:13.562021 23012 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:13.564328 23033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:13.565634 23012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:13.565786 23012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "e570e4d8f1734a13b84bbfd395e5a92e"
format_stamp: "Formatted at 2026-08-12 19:03:13 on dist-test-slave-6nmv"
server_key: "92a1476998d8c88dcf28023c69e7222a"
server_key_iv: "4899b1c9a53c8c0b66b2054ec3c84c2d"
server_key_version: "encryptionkey@0"
I20260812 19:03:13.565935 23012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:13.580725 23012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:13.584075 23012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:13.584301 23012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:13.592391 23012 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34207
I20260812 19:03:13.592417 23096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34207 every 8 connection(s)
I20260812 19:03:13.593402 23012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:13.596247 23097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:13.598328 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 23012
I20260812 19:03:13.598500 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:13.598832 30147 external_mini_cluster.cc:1428] Setting key b88b6d43b2f2e2a7e502281643cd0800
I20260812 19:03:13.603644 23097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e: Bootstrap starting.
I20260812 19:03:13.606427 23097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:13.607344 23097 log.cc:824] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e: Log is configured to *not* fsync() on all Append() calls
Aug 12 19:03:13 dist-test-slave-6nmv krb5kdc[22995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 19:03:13.609412 23097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e: No bootstrap required, opened a new log
I20260812 19:03:13.613405 23097 raft_consensus.cc:348] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570e4d8f1734a13b84bbfd395e5a92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34207 } }
I20260812 19:03:13.613698 23097 raft_consensus.cc:374] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:13.613842 23097 raft_consensus.cc:729] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e570e4d8f1734a13b84bbfd395e5a92e, State: Initialized, Role: FOLLOWER
I20260812 19:03:13.614331 23097 consensus_queue.cc:260] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570e4d8f1734a13b84bbfd395e5a92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34207 } }
I20260812 19:03:13.614511 23097 raft_consensus.cc:388] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:13.614604 23097 raft_consensus.cc:482] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:13.614732 23097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:13.614965 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:13.600477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:43167 (local address 127.0.0.1:34207)
0812 19:03:13.600905 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:13.600922 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:13.600959 (+ 37us) server_negotiation.cc:407] Connection header received
0812 19:03:13.601641 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:13.601658 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:13.602046 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:13.602333 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:13.603310 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:13.604717 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:13.605787 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:13.606355 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:13.606551 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:13.609460 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:13.609466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:13.609479 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:13.609530 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:13.611413 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:13.612018 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:13.612024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:13.612035 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:13.612155 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:13.612536 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:13.612541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:13.612543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:13.612968 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:13.613222 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:13.613577 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 19:03:13.613872 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":137,"threads_started":1}
I20260812 19:03:13.615851 23097 raft_consensus.cc:504] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570e4d8f1734a13b84bbfd395e5a92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34207 } }
I20260812 19:03:13.616237 23097 leader_election.cc:302] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e570e4d8f1734a13b84bbfd395e5a92e; no voters:
I20260812 19:03:13.616640 23097 leader_election.cc:288] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:13.616724 23103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:13.616941 23103 raft_consensus.cc:686] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [term 1 LEADER]: Becoming Leader. State: Replica: e570e4d8f1734a13b84bbfd395e5a92e, State: Running, Role: LEADER
I20260812 19:03:13.617605 23103 consensus_queue.cc:237] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570e4d8f1734a13b84bbfd395e5a92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34207 } }
I20260812 19:03:13.618147 23097 sys_catalog.cc:564] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:13.618309 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:13.619522 23107 sys_catalog.cc:455] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e570e4d8f1734a13b84bbfd395e5a92e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570e4d8f1734a13b84bbfd395e5a92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34207 } } }
I20260812 19:03:13.619660 23107 sys_catalog.cc:458] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:13.620055 23104 sys_catalog.cc:455] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e570e4d8f1734a13b84bbfd395e5a92e. Latest consensus state: current_term: 1 leader_uuid: "e570e4d8f1734a13b84bbfd395e5a92e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e570e4d8f1734a13b84bbfd395e5a92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34207 } } }
I20260812 19:03:13.620198 23104 sys_catalog.cc:458] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:13.620509 23122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:13.624069 23122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:13.629927 23122 catalog_manager.cc:1347] Generated new cluster ID: 13b759b2de1049f98a1443b6b110aecb
I20260812 19:03:13.630075 23122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:13.632390 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:13.620976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:36239 (local address 127.0.0.1:34207)
0812 19:03:13.621147 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:13.621152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:13.621695 (+ 543us) server_negotiation.cc:407] Connection header received
0812 19:03:13.621904 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:13.621907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:13.621962 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:13.622053 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:13.622959 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:13.623839 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:13.625138 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:13.625546 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:13.625632 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:13.627828 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:13.627833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:13.627836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:13.627865 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:13.629933 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:13.630589 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:13.630593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:13.630595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:13.630689 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:13.631173 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:13.631177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:13.631179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:13.631424 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:13.631586 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:13.632108 (+ 522us) server_negotiation.cc:299] Negotiation successful
0812 19:03:13.632222 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 19:03:13.641436 23122 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:13.642465 23122 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:13.647186 23122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e570e4d8f1734a13b84bbfd395e5a92e: Generated new TSK 0
I20260812 19:03:13.647938 23122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:13.656554 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 23012
W20260812 19:03:13.664814 22992 connection.cc:535] client connection to 127.0.0.1:34207 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34207): Connection reset by peer (error 104)
2026-08-12T19:03:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:13 dist-test-slave-6nmv krb5kdc[23136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:13 dist-test-slave-6nmv krb5kdc[23136](info): set up 2 sockets
Aug 12 19:03:13 dist-test-slave-6nmv krb5kdc[23136](info): commencing operation
krb5kdc: starting...
W20260812 19:03:14.812129 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:14 dist-test-slave-6nmv krb5kdc[23136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:03:14.940721 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36331
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:15.055408 23151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:15.055730 23151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:15.055832 23151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:15.059859 23151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:15.059954 23151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:15.059991 23151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:15.060030 23151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:15.060068 23151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:15.065379 23151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36331
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:15.066542 23151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:15.067585 23151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:15.076174 23157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:15.076182 23156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:15.076185 23159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:15.076268 23151 server_base.cc:1034] running on GCE node
I20260812 19:03:15.077042 23151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:15.078243 23151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:15.079442 23151 hybrid_clock.cc:648] HybridClock initialized: now 1786561395079432 us; error 25 us; skew 500 ppm
Aug 12 19:03:15 dist-test-slave-6nmv krb5kdc[23136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561395, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:15.082127 23151 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:15.083086 23151 webserver.cc:460] Webserver started at http://127.0.0.1:38683/ using document root <none> and password file <none>
I20260812 19:03:15.083655 23151 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:15.083760 23151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:15.083992 23151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:15.086129 23151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "84464bc176c64c0d9b39e3169827904c"
format_stamp: "Formatted at 2026-08-12 19:03:15 on dist-test-slave-6nmv"
server_key: "5f39954804f256ac62bfa757ff9111f5"
server_key_iv: "957b31a49b08ceef7f3a4797dde7c0fc"
server_key_version: "encryptionkey@0"
I20260812 19:03:15.086710 23151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "84464bc176c64c0d9b39e3169827904c"
format_stamp: "Formatted at 2026-08-12 19:03:15 on dist-test-slave-6nmv"
server_key: "5f39954804f256ac62bfa757ff9111f5"
server_key_iv: "957b31a49b08ceef7f3a4797dde7c0fc"
server_key_version: "encryptionkey@0"
I20260812 19:03:15.090409 23151 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 19:03:15.092763 23169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:15.093987 23151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:03:15.094126 23151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "84464bc176c64c0d9b39e3169827904c"
format_stamp: "Formatted at 2026-08-12 19:03:15 on dist-test-slave-6nmv"
server_key: "5f39954804f256ac62bfa757ff9111f5"
server_key_iv: "957b31a49b08ceef7f3a4797dde7c0fc"
server_key_version: "encryptionkey@0"
I20260812 19:03:15.094238 23151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:15.122442 23151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:15.125602 23151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:15.125857 23151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:15.133179 23151 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36331
I20260812 19:03:15.133208 23232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36331 every 8 connection(s)
I20260812 19:03:15.134166 23151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:15.136919 23233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:15.137650 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 23151
I20260812 19:03:15.137799 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:15.138144 30147 external_mini_cluster.cc:1428] Setting key 7513bf622ed87c8648958d7dd5bb3bdf
I20260812 19:03:15.143167 23233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c: Bootstrap starting.
I20260812 19:03:15.145458 23233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:15.146273 23233 log.cc:824] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:15.147379 23237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:15.139610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:59919 (local address 127.0.0.1:36331)
0812 19:03:15.140063 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:15.140077 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:15.140111 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:03:15.140769 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:15.140786 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:15.141117 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:15.141446 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:15.141899 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:15.142877 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:15.143842 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:15.144304 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:15.144479 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:15.145411 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:15.145416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:15.145429 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:15.145939 (+ 510us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:15.145998 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:15.146023 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:15.146164 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:15.146347 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 19:03:15.146491 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":132,"threads_started":1}
I20260812 19:03:15.148540 23233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c: No bootstrap required, opened a new log
I20260812 19:03:15.150506 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:15.152815 23233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84464bc176c64c0d9b39e3169827904c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } }
I20260812 19:03:15.153179 23233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:15.153261 23233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84464bc176c64c0d9b39e3169827904c, State: Initialized, Role: FOLLOWER
I20260812 19:03:15.153733 23233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84464bc176c64c0d9b39e3169827904c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } }
I20260812 19:03:15.153957 23233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:15.154035 23233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:15.154143 23233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:15.155464 23233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84464bc176c64c0d9b39e3169827904c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } }
I20260812 19:03:15.155886 23233 leader_election.cc:302] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84464bc176c64c0d9b39e3169827904c; no voters:
I20260812 19:03:15.156319 23233 leader_election.cc:288] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:15.156450 23248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:15.156713 23248 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [term 1 LEADER]: Becoming Leader. State: Replica: 84464bc176c64c0d9b39e3169827904c, State: Running, Role: LEADER
I20260812 19:03:15.157079 23248 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84464bc176c64c0d9b39e3169827904c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } }
I20260812 19:03:15.157429 23237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:15.152631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:59757 (local address 127.0.0.1:36331)
0812 19:03:15.152756 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:15.152760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:15.152893 (+ 133us) server_negotiation.cc:407] Connection header received
0812 19:03:15.153022 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:15.153024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:15.153066 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:15.153132 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:15.153524 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:15.154300 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:15.155423 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:15.155739 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:15.155819 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:15.156879 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:15.156883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:15.156885 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:15.156898 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:15.156928 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:15.156941 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:15.156999 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:15.157263 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 19:03:15.157302 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 19:03:15.157780 23233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:15.159085 23249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84464bc176c64c0d9b39e3169827904c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84464bc176c64c0d9b39e3169827904c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } } }
I20260812 19:03:15.159227 23249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:15.159613 23250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84464bc176c64c0d9b39e3169827904c. Latest consensus state: current_term: 1 leader_uuid: "84464bc176c64c0d9b39e3169827904c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84464bc176c64c0d9b39e3169827904c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36331 } } }
I20260812 19:03:15.159683 23257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:15.159772 23250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:15.162885 23257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:15.168817 23257 catalog_manager.cc:1347] Generated new cluster ID: fd9a74d760d84f7181e0ae3a279cab40
I20260812 19:03:15.168896 23257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:15.193602 23257 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:15.194671 23257 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:15.202250 23257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84464bc176c64c0d9b39e3169827904c: Generated new TSK 0
I20260812 19:03:15.202970 23257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:15.214395 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 23151
W20260812 19:03:15.224603 23134 connection.cc:535] client connection to 127.0.0.1:36331 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36331): Connection reset by peer (error 104)
2026-08-12T19:03:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:15 dist-test-slave-6nmv krb5kdc[23277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:15 dist-test-slave-6nmv krb5kdc[23277](info): set up 2 sockets
Aug 12 19:03:15 dist-test-slave-6nmv krb5kdc[23277](info): commencing operation
krb5kdc: starting...
W20260812 19:03:16.342983 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:16 dist-test-slave-6nmv krb5kdc[23277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 19:03:16.473618 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:16.587754 23292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:16.588049 23292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:16.588141 23292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:16.592182 23292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:16.592267 23292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:16.592304 23292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:16.592365 23292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:16.592396 23292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:16.597256 23292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:16.598295 23292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:16.599347 23292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:16.610611 23303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:16.610913 23292 server_base.cc:1034] running on GCE node
W20260812 19:03:16.610929 23300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:16.614287 23301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:16.615110 23292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:16.616643 23292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:16.617837 23292 hybrid_clock.cc:648] HybridClock initialized: now 1786561396617796 us; error 37 us; skew 500 ppm
Aug 12 19:03:16 dist-test-slave-6nmv krb5kdc[23277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561396, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:16.626713 23292 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 19:03:16.627676 23292 webserver.cc:460] Webserver started at http://127.0.0.1:38755/ using document root <none> and password file <none>
I20260812 19:03:16.628270 23292 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:16.628333 23292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:16.628559 23292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:16.630746 23292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "c041ebad854f4512afa6aa21d99ee3a1"
format_stamp: "Formatted at 2026-08-12 19:03:16 on dist-test-slave-6nmv"
server_key: "860725ba75ac3bb1100ff1ba495df893"
server_key_iv: "d36e781699bc84c55151353f2c088b06"
server_key_version: "encryptionkey@0"
I20260812 19:03:16.631323 23292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "c041ebad854f4512afa6aa21d99ee3a1"
format_stamp: "Formatted at 2026-08-12 19:03:16 on dist-test-slave-6nmv"
server_key: "860725ba75ac3bb1100ff1ba495df893"
server_key_iv: "d36e781699bc84c55151353f2c088b06"
server_key_version: "encryptionkey@0"
I20260812 19:03:16.635125 23292 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:03:16.637516 23310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:16.638809 23292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:16.638964 23292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "c041ebad854f4512afa6aa21d99ee3a1"
format_stamp: "Formatted at 2026-08-12 19:03:16 on dist-test-slave-6nmv"
server_key: "860725ba75ac3bb1100ff1ba495df893"
server_key_iv: "d36e781699bc84c55151353f2c088b06"
server_key_version: "encryptionkey@0"
I20260812 19:03:16.639096 23292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:16.643762 23292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:16.646798 23292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:16.647018 23292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:16.654534 23292 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45821
I20260812 19:03:16.654577 23372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45821 every 8 connection(s)
I20260812 19:03:16.655480 23292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:16.658217 23373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:16.660782 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 23292
I20260812 19:03:16.660936 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:16.661243 30147 external_mini_cluster.cc:1428] Setting key ac2d0f905f86119b3a25db906377d2b9
I20260812 19:03:16.664263 23373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1: Bootstrap starting.
I20260812 19:03:16.666519 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:16.662686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:60643 (local address 127.0.0.1:45821)
0812 19:03:16.663150 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:16.663165 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:16.663201 (+ 36us) server_negotiation.cc:407] Connection header received
0812 19:03:16.663814 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:16.663831 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:16.664099 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:16.664459 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:16.665267 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:16.665271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:16.665283 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:16.665293 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:16.665352 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:16.665371 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:16.665531 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:16.665601 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 19:03:16.665766 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":146,"threads_started":1}
I20260812 19:03:16.666826 23373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:16.667634 23373 log.cc:824] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:16.669587 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:16.669989 23373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1: No bootstrap required, opened a new log
I20260812 19:03:16.673803 23373 raft_consensus.cc:348] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c041ebad854f4512afa6aa21d99ee3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } }
I20260812 19:03:16.674160 23373 raft_consensus.cc:374] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:16.674232 23373 raft_consensus.cc:729] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c041ebad854f4512afa6aa21d99ee3a1, State: Initialized, Role: FOLLOWER
I20260812 19:03:16.674796 23373 consensus_queue.cc:260] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c041ebad854f4512afa6aa21d99ee3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } }
I20260812 19:03:16.674971 23373 raft_consensus.cc:388] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:16.675048 23373 raft_consensus.cc:482] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:16.675160 23373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:16.675745 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:16.671843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:52411 (local address 127.0.0.1:45821)
0812 19:03:16.671957 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:16.671960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:16.674158 (+ 2198us) server_negotiation.cc:407] Connection header received
0812 19:03:16.674339 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:16.674343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:16.674382 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:16.674464 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:16.675290 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:16.675293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:16.675296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:16.675308 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:16.675343 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:16.675354 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:16.675428 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:16.675571 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 19:03:16.675605 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 19:03:16.676453 23373 raft_consensus.cc:504] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c041ebad854f4512afa6aa21d99ee3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } }
I20260812 19:03:16.676806 23373 leader_election.cc:302] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c041ebad854f4512afa6aa21d99ee3a1; no voters:
I20260812 19:03:16.677150 23373 leader_election.cc:288] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:16.677261 23385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:16.677502 23385 raft_consensus.cc:686] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [term 1 LEADER]: Becoming Leader. State: Replica: c041ebad854f4512afa6aa21d99ee3a1, State: Running, Role: LEADER
I20260812 19:03:16.677866 23385 consensus_queue.cc:237] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c041ebad854f4512afa6aa21d99ee3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } }
I20260812 19:03:16.678455 23373 sys_catalog.cc:564] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:16.679527 23387 sys_catalog.cc:455] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c041ebad854f4512afa6aa21d99ee3a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c041ebad854f4512afa6aa21d99ee3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } } }
I20260812 19:03:16.679653 23387 sys_catalog.cc:458] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:16.680060 23397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:16.680006 23388 sys_catalog.cc:455] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c041ebad854f4512afa6aa21d99ee3a1. Latest consensus state: current_term: 1 leader_uuid: "c041ebad854f4512afa6aa21d99ee3a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c041ebad854f4512afa6aa21d99ee3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } } }
I20260812 19:03:16.680300 23388 sys_catalog.cc:458] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:16.683305 23397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:16.688931 23397 catalog_manager.cc:1347] Generated new cluster ID: 9eab8478e8384669a21bf2222a100d96
I20260812 19:03:16.689018 23397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:16.712657 23397 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:16.713001 23397 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:16.720973 23397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c041ebad854f4512afa6aa21d99ee3a1: Generated new TSK 0
I20260812 19:03:16.721783 23397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:16.733107 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 23292
2026-08-12T19:03:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:16 dist-test-slave-6nmv krb5kdc[23410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:16 dist-test-slave-6nmv krb5kdc[23410](info): set up 2 sockets
Aug 12 19:03:16 dist-test-slave-6nmv krb5kdc[23410](info): commencing operation
krb5kdc: starting...
W20260812 19:03:17.866971 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:17 dist-test-slave-6nmv krb5kdc[23410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:03:17.991181 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38407
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:42575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:38407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:18.104202 23425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:18.104528 23425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:18.104625 23425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:18.108454 23425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:18.108538 23425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:18.108572 23425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:18.108614 23425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:18.108649 23425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:18.113899 23425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:42575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:38407
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:18.114959 23425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:18.116008 23425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:18.124456 23430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:18.124452 23431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:18.124466 23433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:18.124719 23425 server_base.cc:1034] running on GCE node
I20260812 19:03:18.125308 23425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:18.126413 23425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:18.127573 23425 hybrid_clock.cc:648] HybridClock initialized: now 1786561398127557 us; error 30 us; skew 500 ppm
Aug 12 19:03:18 dist-test-slave-6nmv krb5kdc[23410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561398, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:18.130367 23425 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:03:18.131232 23425 webserver.cc:460] Webserver started at http://127.29.112.254:38135/ using document root <none> and password file <none>
I20260812 19:03:18.131794 23425 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:18.131856 23425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:18.132068 23425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:18.134116 23425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "710bb31da0264414a799d87dcafccf0d"
format_stamp: "Formatted at 2026-08-12 19:03:18 on dist-test-slave-6nmv"
server_key: "7e6ae40047194d5923e6e15af60c077b"
server_key_iv: "0c03670c64db04d96ce7952e3b72cea8"
server_key_version: "encryptionkey@0"
I20260812 19:03:18.134680 23425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "710bb31da0264414a799d87dcafccf0d"
format_stamp: "Formatted at 2026-08-12 19:03:18 on dist-test-slave-6nmv"
server_key: "7e6ae40047194d5923e6e15af60c077b"
server_key_iv: "0c03670c64db04d96ce7952e3b72cea8"
server_key_version: "encryptionkey@0"
I20260812 19:03:18.138348 23425 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 19:03:18.140648 23440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:18.141785 23425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:03:18.141959 23425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "710bb31da0264414a799d87dcafccf0d"
format_stamp: "Formatted at 2026-08-12 19:03:18 on dist-test-slave-6nmv"
server_key: "7e6ae40047194d5923e6e15af60c077b"
server_key_iv: "0c03670c64db04d96ce7952e3b72cea8"
server_key_version: "encryptionkey@0"
I20260812 19:03:18.142057 23425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:18.159552 23425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:18.162840 23425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:18.163058 23425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:18.170149 23425 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:38407
I20260812 19:03:18.170177 23501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:38407 every 8 connection(s)
I20260812 19:03:18.171087 23425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:18.173791 23502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:18.177735 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 23425
I20260812 19:03:18.177911 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:18.178238 30147 external_mini_cluster.cc:1428] Setting key 5440ce2a6d33677309cccb70dc262d51
I20260812 19:03:18.179955 23502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d: Bootstrap starting.
I20260812 19:03:18.182698 23502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:18.183602 23502 log.cc:824] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:18.185680 23502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d: No bootstrap required, opened a new log
Aug 12 19:03:18 dist-test-slave-6nmv krb5kdc[23410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:03:18.189767 23502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710bb31da0264414a799d87dcafccf0d" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38407 } }
I20260812 19:03:18.190061 23502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:18.190143 23502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710bb31da0264414a799d87dcafccf0d, State: Initialized, Role: FOLLOWER
I20260812 19:03:18.190570 23502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710bb31da0264414a799d87dcafccf0d" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38407 } }
I20260812 19:03:18.190692 23502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:18.190783 23502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:18.190924 23502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:18.192168 23502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710bb31da0264414a799d87dcafccf0d" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38407 } }
I20260812 19:03:18.192519 23502 leader_election.cc:302] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 710bb31da0264414a799d87dcafccf0d; no voters:
I20260812 19:03:18.192924 23502 leader_election.cc:288] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:18.193298 23510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:18.193511 23510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [term 1 LEADER]: Becoming Leader. State: Replica: 710bb31da0264414a799d87dcafccf0d, State: Running, Role: LEADER
I20260812 19:03:18.193892 23510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710bb31da0264414a799d87dcafccf0d" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38407 } }
I20260812 19:03:18.194311 23502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:18.194317 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:18.179696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34981 (local address 127.29.112.254:38407)
0812 19:03:18.180150 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:18.180169 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:18.180215 (+ 46us) server_negotiation.cc:407] Connection header received
0812 19:03:18.181020 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:18.181044 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:18.181498 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:18.181870 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:18.182610 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:18.184111 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:18.185011 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:18.185675 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:18.185866 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:18.188819 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:18.188824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:18.188837 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:18.188863 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:18.191001 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:18.191582 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:18.191587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:18.191591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:18.191707 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:18.192073 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:18.192075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:18.192077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:18.192516 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:18.192716 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:18.193068 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 19:03:18.193331 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":122,"threads_started":1}
I20260812 19:03:18.195613 23509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 710bb31da0264414a799d87dcafccf0d. Latest consensus state: current_term: 1 leader_uuid: "710bb31da0264414a799d87dcafccf0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710bb31da0264414a799d87dcafccf0d" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38407 } } }
I20260812 19:03:18.195737 23509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:18.196054 23512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "710bb31da0264414a799d87dcafccf0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710bb31da0264414a799d87dcafccf0d" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38407 } } }
I20260812 19:03:18.196142 23512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:18.196298 23522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:18.197674 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:18.199713 23522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:18.205622 23522 catalog_manager.cc:1347] Generated new cluster ID: 71d9484297d34dbd9ddf9322c62100cb
I20260812 19:03:18.206066 23522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:18.208773 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:18.199796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35097 (local address 127.29.112.254:38407)
0812 19:03:18.199903 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:18.199907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:18.200246 (+ 339us) server_negotiation.cc:407] Connection header received
0812 19:03:18.200388 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:18.200391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:18.200432 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:18.200525 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:18.201200 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:18.201749 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:18.202953 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:18.203292 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:18.203399 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:18.205108 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:18.205111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:18.205114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:18.205137 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:18.206911 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:18.207480 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:18.207483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:18.207485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:18.207569 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:18.207918 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:18.207921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:18.207922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:18.208106 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:18.208229 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:18.208540 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 19:03:18.208627 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 19:03:18.221982 23522 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:18.222901 23522 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:18.232441 23522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 710bb31da0264414a799d87dcafccf0d: Generated new TSK 0
I20260812 19:03:18.233107 23522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:18.238559 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 23425
W20260812 19:03:18.249181 23408 connection.cc:535] client connection to 127.29.112.254:38407 recv error: Network error: failed to read from TLS socket (remote: 127.29.112.254:38407): Connection reset by peer (error 104)
2026-08-12T19:03:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:18 dist-test-slave-6nmv krb5kdc[23540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:18 dist-test-slave-6nmv krb5kdc[23540](info): set up 2 sockets
Aug 12 19:03:18 dist-test-slave-6nmv krb5kdc[23540](info): commencing operation
krb5kdc: starting...
W20260812 19:03:19.367228 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:19 dist-test-slave-6nmv krb5kdc[23540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:03:19.490953 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:40479
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:42897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:40479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:19.601245 23558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:19.601547 23558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:19.601626 23558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:19.605440 23558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:19.605528 23558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:19.605563 23558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:19.605618 23558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:19.605654 23558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:19.610759 23558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:42897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:40479
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:40479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:19.611821 23558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:19.612824 23558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:19.621481 23566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:19.621493 23564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:19.621558 23558 server_base.cc:1034] running on GCE node
W20260812 19:03:19.621500 23568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:19.622330 23558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:19.623376 23558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:19.624562 23558 hybrid_clock.cc:648] HybridClock initialized: now 1786561399624545 us; error 28 us; skew 500 ppm
Aug 12 19:03:19 dist-test-slave-6nmv krb5kdc[23540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561399, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:19.627398 23558 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:03:19.628260 23558 webserver.cc:460] Webserver started at http://127.29.112.254:46459/ using document root <none> and password file <none>
I20260812 19:03:19.628754 23558 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:19.628810 23558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:19.628965 23558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:19.630972 23558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "76c99a0c1a9f4309bfe7a21bf6322887"
format_stamp: "Formatted at 2026-08-12 19:03:19 on dist-test-slave-6nmv"
server_key: "84df0b43b7a170f2acfb15dff19f0275"
server_key_iv: "95588db85ee07f1efc82b8864432c2b0"
server_key_version: "encryptionkey@0"
I20260812 19:03:19.631461 23558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "76c99a0c1a9f4309bfe7a21bf6322887"
format_stamp: "Formatted at 2026-08-12 19:03:19 on dist-test-slave-6nmv"
server_key: "84df0b43b7a170f2acfb15dff19f0275"
server_key_iv: "95588db85ee07f1efc82b8864432c2b0"
server_key_version: "encryptionkey@0"
I20260812 19:03:19.635068 23558 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 19:03:19.637374 23577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:19.638638 23558 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 19:03:19.638733 23558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "76c99a0c1a9f4309bfe7a21bf6322887"
format_stamp: "Formatted at 2026-08-12 19:03:19 on dist-test-slave-6nmv"
server_key: "84df0b43b7a170f2acfb15dff19f0275"
server_key_iv: "95588db85ee07f1efc82b8864432c2b0"
server_key_version: "encryptionkey@0"
I20260812 19:03:19.638814 23558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:19.657179 23558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:19.660171 23558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:19.660327 23558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:19.667685 23558 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:40479
I20260812 19:03:19.667699 23642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:40479 every 8 connection(s)
I20260812 19:03:19.668653 23558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:19.671478 23643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:19.676882 23643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887: Bootstrap starting.
I20260812 19:03:19.677568 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 23558
I20260812 19:03:19.677742 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:19.678050 30147 external_mini_cluster.cc:1428] Setting key aef521699d8b5ad886d13ff5dbb5285f
I20260812 19:03:19.679694 23643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:19.680447 23643 log.cc:824] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:19.682524 23643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887: No bootstrap required, opened a new log
I20260812 19:03:19.686328 23643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c99a0c1a9f4309bfe7a21bf6322887" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40479 } }
I20260812 19:03:19.686595 23643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:19.686681 23643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c99a0c1a9f4309bfe7a21bf6322887, State: Initialized, Role: FOLLOWER
I20260812 19:03:19.687040 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:19.679185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36897 (local address 127.29.112.254:40479)
0812 19:03:19.679591 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:19.679606 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:19.679641 (+ 35us) server_negotiation.cc:407] Connection header received
0812 19:03:19.680302 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:19.680320 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:19.680705 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:19.681072 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:19.681456 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:19.682510 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:19.683421 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:19.683892 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:19.684062 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:19.685074 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:19.685079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:19.685090 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:19.685472 (+ 382us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:19.685536 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:19.685560 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:19.685707 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:19.685937 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 19:03:19.686070 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":134,"threads_started":1}
I20260812 19:03:19.687175 23643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c99a0c1a9f4309bfe7a21bf6322887" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40479 } }
I20260812 19:03:19.687351 23643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:19.687443 23643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:19.687547 23643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:19.688467 23643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c99a0c1a9f4309bfe7a21bf6322887" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40479 } }
I20260812 19:03:19.688827 23643 leader_election.cc:302] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76c99a0c1a9f4309bfe7a21bf6322887; no voters:
I20260812 19:03:19.689219 23643 leader_election.cc:288] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:19.689330 23652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:19.689556 23652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [term 1 LEADER]: Becoming Leader. State: Replica: 76c99a0c1a9f4309bfe7a21bf6322887, State: Running, Role: LEADER
I20260812 19:03:19.689918 23652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c99a0c1a9f4309bfe7a21bf6322887" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40479 } }
I20260812 19:03:19.690062 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:19.690541 23643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:19.693069 23654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76c99a0c1a9f4309bfe7a21bf6322887. Latest consensus state: current_term: 1 leader_uuid: "76c99a0c1a9f4309bfe7a21bf6322887" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c99a0c1a9f4309bfe7a21bf6322887" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40479 } } }
I20260812 19:03:19.693202 23654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:19.693077 23653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76c99a0c1a9f4309bfe7a21bf6322887" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c99a0c1a9f4309bfe7a21bf6322887" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 40479 } } }
I20260812 19:03:19.693522 23653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887 [sys.catalog]: This master's current role is: LEADER
W20260812 19:03:19.696625 23676 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:03:19.696710 23676 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:19.696805 23678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:19.697736 23678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:19.698086 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:19.693371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47905 (local address 127.29.112.254:40479)
0812 19:03:19.693579 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:19.693584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:19.693598 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:03:19.693758 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:19.693764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:19.693838 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:19.693926 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:19.694304 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:19.694805 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:19.695922 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:19.696293 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:19.696403 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:19.697485 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:19.697489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:19.697492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:19.697506 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:19.697537 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:19.697551 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:19.697650 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:19.697904 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 19:03:19.697942 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260812 19:03:19.703397 23678 catalog_manager.cc:1347] Generated new cluster ID: 4728fc6be2fe456fa19318fb388570ee
I20260812 19:03:19.703480 23678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:19.712633 23678 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:19.713544 23678 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:19.734169 23678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76c99a0c1a9f4309bfe7a21bf6322887: Generated new TSK 0
I20260812 19:03:19.734956 23678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:19.743140 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 23558
W20260812 19:03:19.753013 23538 connection.cc:535] client connection to 127.29.112.254:40479 recv error: Network error: failed to read from TLS socket (remote: 127.29.112.254:40479): Connection reset by peer (error 104)
2026-08-12T19:03:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:19 dist-test-slave-6nmv krb5kdc[23684](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:19 dist-test-slave-6nmv krb5kdc[23684](info): set up 2 sockets
Aug 12 19:03:19 dist-test-slave-6nmv krb5kdc[23684](info): commencing operation
krb5kdc: starting...
W20260812 19:03:20.877977 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:20 dist-test-slave-6nmv krb5kdc[23684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:03:21.005509 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:35639
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:35639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:21.119551 23705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:21.119855 23705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:21.119932 23705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:21.123880 23705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:21.123965 23705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:21.124001 23705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:21.124058 23705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:21.124096 23705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:21.129149 23705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:35639
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:35639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:21.130275 23705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:21.131268 23705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:21.139617 23713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:21.139569 23712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:21.139654 23717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:21.139739 23705 server_base.cc:1034] running on GCE node
I20260812 19:03:21.140380 23705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:21.141443 23705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:21.142619 23705 hybrid_clock.cc:648] HybridClock initialized: now 1786561401142586 us; error 31 us; skew 500 ppm
Aug 12 19:03:21 dist-test-slave-6nmv krb5kdc[23684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561401, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:21.145293 23705 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:03:21.146289 23705 webserver.cc:460] Webserver started at http://127.29.112.254:44213/ using document root <none> and password file <none>
I20260812 19:03:21.146840 23705 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:21.146915 23705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:21.147126 23705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:21.149271 23705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "68a2267061bc455f80c0c7ff51fe4e77"
format_stamp: "Formatted at 2026-08-12 19:03:21 on dist-test-slave-6nmv"
server_key: "0e06a6e0cb0d2393078f4b39f7eb2407"
server_key_iv: "b8ce57dcca5e02e1ba57bd6af323f2bd"
server_key_version: "encryptionkey@0"
I20260812 19:03:21.149936 23705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "68a2267061bc455f80c0c7ff51fe4e77"
format_stamp: "Formatted at 2026-08-12 19:03:21 on dist-test-slave-6nmv"
server_key: "0e06a6e0cb0d2393078f4b39f7eb2407"
server_key_iv: "b8ce57dcca5e02e1ba57bd6af323f2bd"
server_key_version: "encryptionkey@0"
I20260812 19:03:21.153736 23705 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 19:03:21.156209 23724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:21.157564 23705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:21.157722 23705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "68a2267061bc455f80c0c7ff51fe4e77"
format_stamp: "Formatted at 2026-08-12 19:03:21 on dist-test-slave-6nmv"
server_key: "0e06a6e0cb0d2393078f4b39f7eb2407"
server_key_iv: "b8ce57dcca5e02e1ba57bd6af323f2bd"
server_key_version: "encryptionkey@0"
I20260812 19:03:21.157868 23705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:21.162596 23705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:21.165453 23705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:21.165668 23705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:21.172883 23705 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:35639
I20260812 19:03:21.172910 23786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:35639 every 8 connection(s)
I20260812 19:03:21.173892 23705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:21.176513 23787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:21.181914 23787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77: Bootstrap starting.
I20260812 19:03:21.182238 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 23705
I20260812 19:03:21.182372 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:21.182679 30147 external_mini_cluster.cc:1428] Setting key 242c8ccae12709b92da56113ddc10e2d
I20260812 19:03:21.184629 23787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:21.185336 23787 log.cc:824] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:21.187631 23787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77: No bootstrap required, opened a new log
I20260812 19:03:21.188097 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:21.184086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52163 (local address 127.29.112.254:35639)
0812 19:03:21.184548 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:21.184566 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:21.184602 (+ 36us) server_negotiation.cc:407] Connection header received
0812 19:03:21.185296 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:21.185313 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:21.185641 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:21.186068 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:21.186861 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:21.186866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:21.186877 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:21.186887 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:21.186940 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:21.186959 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:21.187090 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:21.187202 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 19:03:21.187366 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":122,"threads_started":1}
I20260812 19:03:21.190860 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:21.191444 23787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a2267061bc455f80c0c7ff51fe4e77" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35639 } }
I20260812 19:03:21.191665 23787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:21.191732 23787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68a2267061bc455f80c0c7ff51fe4e77, State: Initialized, Role: FOLLOWER
I20260812 19:03:21.192209 23787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a2267061bc455f80c0c7ff51fe4e77" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35639 } }
I20260812 19:03:21.192359 23787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:21.192440 23787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:21.192555 23787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:21.193723 23787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a2267061bc455f80c0c7ff51fe4e77" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35639 } }
I20260812 19:03:21.194109 23787 leader_election.cc:302] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68a2267061bc455f80c0c7ff51fe4e77; no voters:
I20260812 19:03:21.194499 23787 leader_election.cc:288] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:21.194586 23799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:21.194809 23799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [term 1 LEADER]: Becoming Leader. State: Replica: 68a2267061bc455f80c0c7ff51fe4e77, State: Running, Role: LEADER
I20260812 19:03:21.194948 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:21.193068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58313 (local address 127.29.112.254:35639)
0812 19:03:21.193197 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:21.193201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:21.193424 (+ 223us) server_negotiation.cc:407] Connection header received
0812 19:03:21.193553 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:21.193555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:21.193582 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:21.193643 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:21.194358 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:21.194362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:21.194364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:21.194383 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:21.194407 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:21.194418 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:21.194532 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:21.194801 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 19:03:21.194841 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 19:03:21.195163 23799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a2267061bc455f80c0c7ff51fe4e77" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35639 } }
I20260812 19:03:21.195760 23787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:21.196400 23799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68a2267061bc455f80c0c7ff51fe4e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a2267061bc455f80c0c7ff51fe4e77" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35639 } } }
I20260812 19:03:21.196517 23799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:21.196718 23799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68a2267061bc455f80c0c7ff51fe4e77. Latest consensus state: current_term: 1 leader_uuid: "68a2267061bc455f80c0c7ff51fe4e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68a2267061bc455f80c0c7ff51fe4e77" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 35639 } } }
I20260812 19:03:21.196805 23799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:21.197109 23809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:21.200232 23809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:21.206102 23809 catalog_manager.cc:1347] Generated new cluster ID: 6eb0264ddabb4cd79f3ea1d30ef9cf9b
I20260812 19:03:21.206194 23809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:21.216835 23809 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:21.217180 23809 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:21.223201 23809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68a2267061bc455f80c0c7ff51fe4e77: Generated new TSK 0
I20260812 19:03:21.223838 23809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:21.229950 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 23705
2026-08-12T19:03:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:21 dist-test-slave-6nmv krb5kdc[23824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:21 dist-test-slave-6nmv krb5kdc[23824](info): set up 2 sockets
Aug 12 19:03:21 dist-test-slave-6nmv krb5kdc[23824](info): commencing operation
krb5kdc: starting...
W20260812 19:03:22.361362 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:22 dist-test-slave-6nmv krb5kdc[23824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:03:22.488515 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:43635
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:43635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:22.599545 23840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:22.599838 23840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:22.599928 23840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:22.603955 23840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:22.604038 23840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:22.604072 23840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:22.604113 23840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:22.604156 23840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:22.610926 23840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:43635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:43635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:22.612016 23840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:22.612994 23840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:22.621855 23851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:22.621871 23847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:22.621932 23846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:22.622138 23840 server_base.cc:1034] running on GCE node
I20260812 19:03:22.622676 23840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:22.623710 23840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:22.624874 23840 hybrid_clock.cc:648] HybridClock initialized: now 1786561402624863 us; error 27 us; skew 500 ppm
Aug 12 19:03:22 dist-test-slave-6nmv krb5kdc[23824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561402, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:22.633141 23840 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:03:22.634208 23840 webserver.cc:460] Webserver started at http://127.29.112.254:44457/ using document root <none> and password file <none>
I20260812 19:03:22.634825 23840 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:22.634888 23840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:22.635138 23840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:22.637238 23840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "9e787bd2f4d440ad929330812a7a64a9"
format_stamp: "Formatted at 2026-08-12 19:03:22 on dist-test-slave-6nmv"
server_key: "f0212dbcb484ea0d71a58c0f7623b133"
server_key_iv: "f8fc50e53602a818f92a06b314cd6b4a"
server_key_version: "encryptionkey@0"
I20260812 19:03:22.637863 23840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "9e787bd2f4d440ad929330812a7a64a9"
format_stamp: "Formatted at 2026-08-12 19:03:22 on dist-test-slave-6nmv"
server_key: "f0212dbcb484ea0d71a58c0f7623b133"
server_key_iv: "f8fc50e53602a818f92a06b314cd6b4a"
server_key_version: "encryptionkey@0"
I20260812 19:03:22.641587 23840 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 19:03:22.644079 23858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:22.645258 23840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:22.645397 23840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "9e787bd2f4d440ad929330812a7a64a9"
format_stamp: "Formatted at 2026-08-12 19:03:22 on dist-test-slave-6nmv"
server_key: "f0212dbcb484ea0d71a58c0f7623b133"
server_key_iv: "f8fc50e53602a818f92a06b314cd6b4a"
server_key_version: "encryptionkey@0"
I20260812 19:03:22.645507 23840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:22.665405 23840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:22.668391 23840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:22.668612 23840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:22.676074 23840 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:43635
I20260812 19:03:22.676102 23923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:43635 every 8 connection(s)
I20260812 19:03:22.677049 23840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:22.679865 23924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:22.685503 23924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9: Bootstrap starting.
I20260812 19:03:22.685670 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 23840
I20260812 19:03:22.685875 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:22.686162 30147 external_mini_cluster.cc:1428] Setting key da0b07969eaec0275b8fa6255c099b19
I20260812 19:03:22.688372 23924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:22.689265 23924 log.cc:824] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:22.691592 23924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9: No bootstrap required, opened a new log
I20260812 19:03:22.695768 23924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e787bd2f4d440ad929330812a7a64a9" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43635 } }
Aug 12 19:03:22 dist-test-slave-6nmv krb5kdc[23824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:03:22.696070 23924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:22.696130 23924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e787bd2f4d440ad929330812a7a64a9, State: Initialized, Role: FOLLOWER
I20260812 19:03:22.696681 23924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e787bd2f4d440ad929330812a7a64a9" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43635 } }
I20260812 19:03:22.696818 23924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:22.696908 23924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:22.697032 23924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:22.698037 23924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e787bd2f4d440ad929330812a7a64a9" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43635 } }
I20260812 19:03:22.698380 23924 leader_election.cc:302] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e787bd2f4d440ad929330812a7a64a9; no voters:
I20260812 19:03:22.698712 23924 leader_election.cc:288] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:22.698845 23930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:22.699076 23930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [term 1 LEADER]: Becoming Leader. State: Replica: 9e787bd2f4d440ad929330812a7a64a9, State: Running, Role: LEADER
I20260812 19:03:22.699445 23930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e787bd2f4d440ad929330812a7a64a9" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43635 } }
I20260812 19:03:22.700078 23924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:22.702790 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:22.687565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39889 (local address 127.29.112.254:43635)
0812 19:03:22.688015 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:22.688043 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:22.688085 (+ 42us) server_negotiation.cc:407] Connection header received
0812 19:03:22.688957 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:22.688979 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:22.689434 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:22.689858 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:22.690681 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:22.692131 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:22.692977 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:22.693694 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:22.693894 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:22.696699 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:22.696704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:22.696718 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:22.696746 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:22.698951 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:22.699512 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:22.699517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:22.699523 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:22.699680 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:22.699987 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:22.699990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:22.699992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:22.700514 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:22.700747 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:22.701545 (+ 798us) server_negotiation.cc:299] Negotiation successful
0812 19:03:22.701857 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
I20260812 19:03:22.703054 23931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e787bd2f4d440ad929330812a7a64a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e787bd2f4d440ad929330812a7a64a9" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43635 } } }
I20260812 19:03:22.704165 23931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:22.703282 23933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e787bd2f4d440ad929330812a7a64a9. Latest consensus state: current_term: 1 leader_uuid: "9e787bd2f4d440ad929330812a7a64a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e787bd2f4d440ad929330812a7a64a9" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 43635 } } }
I20260812 19:03:22.704758 23933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9 [sys.catalog]: This master's current role is: LEADER
W20260812 19:03:22.705881 23948 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:03:22.705955 23948 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:22.706041 23950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:22.706159 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:22.706867 23950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:22.712845 23950 catalog_manager.cc:1347] Generated new cluster ID: 57901d72eb2f462ca04cf7e4af00fc20
I20260812 19:03:22.712908 23950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:22.719077 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:22.709153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36137 (local address 127.29.112.254:43635)
0812 19:03:22.709943 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:22.709947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:22.709961 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:03:22.709997 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:22.710000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:22.710042 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:22.710136 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:22.710988 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:22.712211 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:22.713406 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:22.713738 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:22.713877 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:22.715561 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:22.715564 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:22.715567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:22.715591 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:22.717284 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:22.717766 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:22.717769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:22.717771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:22.717876 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:22.718176 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:22.718180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:22.718182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:22.718415 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:22.718564 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:22.718786 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 19:03:22.718904 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715}
I20260812 19:03:22.737218 23950 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:22.738560 23950 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:22.762316 23950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e787bd2f4d440ad929330812a7a64a9: Generated new TSK 0
I20260812 19:03:22.763130 23950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:22.775921 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 23840
2026-08-12T19:03:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:22 dist-test-slave-6nmv krb5kdc[23960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:22 dist-test-slave-6nmv krb5kdc[23960](info): set up 2 sockets
Aug 12 19:03:22 dist-test-slave-6nmv krb5kdc[23960](info): commencing operation
krb5kdc: starting...
W20260812 19:03:23.964468 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:24 dist-test-slave-6nmv krb5kdc[23960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:03:24.092877 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:42539
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:42539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:24.203457 23978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:24.203740 23978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:24.203783 23978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:24.207659 23978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:24.207711 23978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:24.207731 23978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:24.207746 23978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:24.207760 23978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:24.212864 23978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:42539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:42539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:24.213825 23978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:24.214753 23978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:03:24.223269 23978 server_base.cc:1034] running on GCE node
W20260812 19:03:24.223222 23984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:24.223241 23987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:24.223217 23985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:24.223984 23978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:24.225003 23978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:24.226156 23978 hybrid_clock.cc:648] HybridClock initialized: now 1786561404226134 us; error 36 us; skew 500 ppm
Aug 12 19:03:24 dist-test-slave-6nmv krb5kdc[23960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561404, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:24.228955 23978 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:03:24.229777 23978 webserver.cc:460] Webserver started at http://127.29.112.254:42525/ using document root <none> and password file <none>
I20260812 19:03:24.230351 23978 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:24.230412 23978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:24.230582 23978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:24.232546 23978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "ed032883aef34416922701a4e03421b3"
format_stamp: "Formatted at 2026-08-12 19:03:24 on dist-test-slave-6nmv"
server_key: "b695b2fc89e33271c73b72813d51ecec"
server_key_iv: "ce4eeeb8fde5cca88129d83908b0a5fb"
server_key_version: "encryptionkey@0"
I20260812 19:03:24.233088 23978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "ed032883aef34416922701a4e03421b3"
format_stamp: "Formatted at 2026-08-12 19:03:24 on dist-test-slave-6nmv"
server_key: "b695b2fc89e33271c73b72813d51ecec"
server_key_iv: "ce4eeeb8fde5cca88129d83908b0a5fb"
server_key_version: "encryptionkey@0"
I20260812 19:03:24.236850 23978 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 19:03:24.239130 23997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:24.240310 23978 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:03:24.240433 23978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "ed032883aef34416922701a4e03421b3"
format_stamp: "Formatted at 2026-08-12 19:03:24 on dist-test-slave-6nmv"
server_key: "b695b2fc89e33271c73b72813d51ecec"
server_key_iv: "ce4eeeb8fde5cca88129d83908b0a5fb"
server_key_version: "encryptionkey@0"
I20260812 19:03:24.240542 23978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:24.252521 23978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:24.255283 23978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:24.255477 23978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:24.262315 23978 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:42539
I20260812 19:03:24.262333 24060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:42539 every 8 connection(s)
I20260812 19:03:24.263379 23978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:24.266319 24061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:24.269404 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 23978
I20260812 19:03:24.269546 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:24.269881 30147 external_mini_cluster.cc:1428] Setting key 9cbf98d6a3c9185bed1158ab177bc6c6
I20260812 19:03:24.272475 24061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3: Bootstrap starting.
I20260812 19:03:24.274712 24061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:24.275429 24061 log.cc:824] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:24.277294 24061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3: No bootstrap required, opened a new log
I20260812 19:03:24.278703 24066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:24.271271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51207 (local address 127.29.112.254:42539)
0812 19:03:24.271716 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:24.271735 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:24.271768 (+ 33us) server_negotiation.cc:407] Connection header received
0812 19:03:24.272353 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:24.272368 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:24.272743 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:24.273066 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:24.273471 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:24.274525 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:24.275473 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:24.275971 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:24.276151 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:24.277024 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:24.277029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:24.277041 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:24.277450 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:24.277512 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:24.277536 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:24.277640 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:24.277783 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 19:03:24.278013 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":143,"threads_started":1}
I20260812 19:03:24.281437 24061 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed032883aef34416922701a4e03421b3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42539 } }
I20260812 19:03:24.281731 24061 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:24.281720 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:24.281859 24061 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed032883aef34416922701a4e03421b3, State: Initialized, Role: FOLLOWER
I20260812 19:03:24.282269 24061 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed032883aef34416922701a4e03421b3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42539 } }
I20260812 19:03:24.282423 24061 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:24.282510 24061 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:24.282618 24061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:24.283680 24061 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed032883aef34416922701a4e03421b3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42539 } }
I20260812 19:03:24.284054 24061 leader_election.cc:302] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed032883aef34416922701a4e03421b3; no voters:
I20260812 19:03:24.284421 24061 leader_election.cc:288] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:24.284487 24076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:24.284698 24076 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [term 1 LEADER]: Becoming Leader. State: Replica: ed032883aef34416922701a4e03421b3, State: Running, Role: LEADER
I20260812 19:03:24.285000 24076 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed032883aef34416922701a4e03421b3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42539 } }
I20260812 19:03:24.285709 24061 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:24.286185 24076 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed032883aef34416922701a4e03421b3. Latest consensus state: current_term: 1 leader_uuid: "ed032883aef34416922701a4e03421b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed032883aef34416922701a4e03421b3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42539 } } }
I20260812 19:03:24.286301 24076 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:24.286506 24076 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed032883aef34416922701a4e03421b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed032883aef34416922701a4e03421b3" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42539 } } }
I20260812 19:03:24.286625 24076 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3 [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:24.286922 24080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:24.288762 24066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:24.283944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33691 (local address 127.29.112.254:42539)
0812 19:03:24.284055 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:24.284059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:24.284392 (+ 333us) server_negotiation.cc:407] Connection header received
0812 19:03:24.284571 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:24.284576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:24.284621 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:24.284717 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:24.285051 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:24.285595 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:24.286714 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:24.287085 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:24.287161 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:24.288229 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:24.288233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:24.288236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:24.288251 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:24.288281 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:24.288294 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:24.288348 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:24.288544 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 19:03:24.288596 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 19:03:24.290391 24080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:24.295542 24080 catalog_manager.cc:1347] Generated new cluster ID: cd5acfcd5de549c0b36cf07dd42c5261
I20260812 19:03:24.295615 24080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:24.320143 24080 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:24.321426 24080 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:24.327555 24080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed032883aef34416922701a4e03421b3: Generated new TSK 0
I20260812 19:03:24.328326 24080 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:24.335985 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 23978
2026-08-12T19:03:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1561 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:24 dist-test-slave-6nmv krb5kdc[24099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:24 dist-test-slave-6nmv krb5kdc[24099](info): set up 2 sockets
Aug 12 19:03:24 dist-test-slave-6nmv krb5kdc[24099](info): commencing operation
krb5kdc: starting...
W20260812 19:03:25.502298 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:25 dist-test-slave-6nmv krb5kdc[24099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:03:25.629582 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38195
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:38195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:25.740155 24115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:25.740466 24115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:25.740541 24115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:25.744531 24115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:25.744637 24115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:25.744678 24115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:25.744737 24115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:25.744777 24115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:25.750005 24115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:38195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:38195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:25.751262 24115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:25.752368 24115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:25.761075 24123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:25.761092 24120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:25.761075 24121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:25.761338 24115 server_base.cc:1034] running on GCE node
I20260812 19:03:25.762033 24115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:25.763068 24115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:25.764240 24115 hybrid_clock.cc:648] HybridClock initialized: now 1786561405764219 us; error 31 us; skew 500 ppm
Aug 12 19:03:25 dist-test-slave-6nmv krb5kdc[24099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561405, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:25.767244 24115 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:03:25.768165 24115 webserver.cc:460] Webserver started at http://127.29.112.254:38191/ using document root <none> and password file <none>
I20260812 19:03:25.768718 24115 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:25.768781 24115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:25.769035 24115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:25.771143 24115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "68e4f70e0c6043c6b73ffded18a5f67e"
format_stamp: "Formatted at 2026-08-12 19:03:25 on dist-test-slave-6nmv"
server_key: "3b774ad8d096517a484f5ea1be857d13"
server_key_iv: "8d0fa3b3f69e325c3dde30ae4525c4b6"
server_key_version: "encryptionkey@0"
I20260812 19:03:25.771729 24115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "68e4f70e0c6043c6b73ffded18a5f67e"
format_stamp: "Formatted at 2026-08-12 19:03:25 on dist-test-slave-6nmv"
server_key: "3b774ad8d096517a484f5ea1be857d13"
server_key_iv: "8d0fa3b3f69e325c3dde30ae4525c4b6"
server_key_version: "encryptionkey@0"
I20260812 19:03:25.775545 24115 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:03:25.777974 24130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:25.779214 24115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:03:25.779361 24115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "68e4f70e0c6043c6b73ffded18a5f67e"
format_stamp: "Formatted at 2026-08-12 19:03:25 on dist-test-slave-6nmv"
server_key: "3b774ad8d096517a484f5ea1be857d13"
server_key_iv: "8d0fa3b3f69e325c3dde30ae4525c4b6"
server_key_version: "encryptionkey@0"
I20260812 19:03:25.779472 24115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:25.784054 24115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:25.787070 24115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:25.787279 24115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:25.794373 24115 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:38195
I20260812 19:03:25.794404 24191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:38195 every 8 connection(s)
I20260812 19:03:25.795292 24115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:25.795939 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 24115
I20260812 19:03:25.796118 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:25.796429 30147 external_mini_cluster.cc:1428] Setting key 115d60f2fabc7b506265748b94af5739
I20260812 19:03:25.798650 24193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:25.801587 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:25.797867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36117 (local address 127.29.112.254:38195)
0812 19:03:25.798327 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:25.798341 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:25.798374 (+ 33us) server_negotiation.cc:407] Connection header received
0812 19:03:25.799058 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:25.799075 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:25.799346 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:25.799691 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:25.800460 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:25.800464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:25.800475 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:25.800485 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:25.800542 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:25.800560 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:25.800716 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:25.800787 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 19:03:25.800938 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":163,"threads_started":1}
I20260812 19:03:25.804461 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:25.804806 24193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e: Bootstrap starting.
I20260812 19:03:25.807241 24193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:25.808229 24193 log.cc:824] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:25.808408 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:25.806556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59955 (local address 127.29.112.254:38195)
0812 19:03:25.806686 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:25.806691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:25.806893 (+ 202us) server_negotiation.cc:407] Connection header received
0812 19:03:25.806996 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:25.806998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:25.807025 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:25.807085 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:25.808007 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:25.808010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:25.808011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 19:03:25.808021 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:25.808046 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 19:03:25.808056 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:25.808125 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:25.808236 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 19:03:25.808266 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 19:03:25.810297 24193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e: No bootstrap required, opened a new log
I20260812 19:03:25.813673 24193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e4f70e0c6043c6b73ffded18a5f67e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38195 } }
I20260812 19:03:25.813967 24193 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:03:25.814028 24193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e4f70e0c6043c6b73ffded18a5f67e, State: Initialized, Role: FOLLOWER
I20260812 19:03:25.814442 24193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e4f70e0c6043c6b73ffded18a5f67e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38195 } }
I20260812 19:03:25.814591 24193 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:03:25.814675 24193 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:03:25.814807 24193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:25.815697 24193 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e4f70e0c6043c6b73ffded18a5f67e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38195 } }
I20260812 19:03:25.816028 24193 leader_election.cc:302] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68e4f70e0c6043c6b73ffded18a5f67e; no voters:
I20260812 19:03:25.816319 24193 leader_election.cc:288] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:03:25.816440 24205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:25.816716 24205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [term 1 LEADER]: Becoming Leader. State: Replica: 68e4f70e0c6043c6b73ffded18a5f67e, State: Running, Role: LEADER
I20260812 19:03:25.817014 24205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e4f70e0c6043c6b73ffded18a5f67e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38195 } }
I20260812 19:03:25.817592 24193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:25.818725 24206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68e4f70e0c6043c6b73ffded18a5f67e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e4f70e0c6043c6b73ffded18a5f67e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38195 } } }
I20260812 19:03:25.818862 24206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:25.819187 24207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68e4f70e0c6043c6b73ffded18a5f67e. Latest consensus state: current_term: 1 leader_uuid: "68e4f70e0c6043c6b73ffded18a5f67e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e4f70e0c6043c6b73ffded18a5f67e" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 38195 } } }
I20260812 19:03:25.819298 24207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e [sys.catalog]: This master's current role is: LEADER
W20260812 19:03:25.822325 24225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:03:25.822387 24225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:25.822467 24216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:25.823115 24216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:25.829169 24216 catalog_manager.cc:1347] Generated new cluster ID: fbe8348ee2fd42ba86cb7bedf9c71e97
I20260812 19:03:25.829232 24216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:25.854128 24216 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:25.854470 24216 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:25.862534 24216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68e4f70e0c6043c6b73ffded18a5f67e: Generated new TSK 0
I20260812 19:03:25.863359 24216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:03:25.872074 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 24115
2026-08-12T19:03:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1534 ms)
[----------] 18 tests from AuthTokenIssuingTest (27216 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:25 dist-test-slave-6nmv krb5kdc[24234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:25 dist-test-slave-6nmv krb5kdc[24234](info): set up 2 sockets
Aug 12 19:03:25 dist-test-slave-6nmv krb5kdc[24234](info): commencing operation
krb5kdc: starting...
W20260812 19:03:27.000581 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:03:27.129853 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:44161
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:44161,127.29.112.253:33981,127.29.112.252:43581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:27.244558 24248 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:03:27.244949 24248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:27.245031 24248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:27.245123 24248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:27.249294 24248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:27.249400 24248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:27.249439 24248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:27.249477 24248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:27.249512 24248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:27.255072 24248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:44161,127.29.112.253:33981,127.29.112.252:43581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:44161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:27.256249 24248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:27.257328 24248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:27.265936 24260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:27.265929 24255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:27.265929 24254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:27.266122 24248 server_base.cc:1034] running on GCE node
I20260812 19:03:27.266901 24248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:27.268258 24248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:27.269482 24248 hybrid_clock.cc:648] HybridClock initialized: now 1786561407269463 us; error 36 us; skew 500 ppm
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:27.272506 24248 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:03:27.273483 24248 webserver.cc:460] Webserver started at http://127.29.112.254:37067/ using document root <none> and password file <none>
I20260812 19:03:27.274142 24248 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:27.274245 24248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:27.274500 24248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:27.276923 24248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c"
format_stamp: "Formatted at 2026-08-12 19:03:27 on dist-test-slave-6nmv"
server_key: "b6b991631cd5cb7be8fdad0cd169911b"
server_key_iv: "2a2ec671f600890be856ed39a115fa34"
server_key_version: "encryptionkey@0"
I20260812 19:03:27.277539 24248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c"
format_stamp: "Formatted at 2026-08-12 19:03:27 on dist-test-slave-6nmv"
server_key: "b6b991631cd5cb7be8fdad0cd169911b"
server_key_iv: "2a2ec671f600890be856ed39a115fa34"
server_key_version: "encryptionkey@0"
I20260812 19:03:27.281499 24248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 19:03:27.284173 24267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:27.285411 24248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:27.285564 24248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c"
format_stamp: "Formatted at 2026-08-12 19:03:27 on dist-test-slave-6nmv"
server_key: "b6b991631cd5cb7be8fdad0cd169911b"
server_key_iv: "2a2ec671f600890be856ed39a115fa34"
server_key_version: "encryptionkey@0"
I20260812 19:03:27.285686 24248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:27.302204 24248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:27.305478 24248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:27.305716 24248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:27.312964 24248 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:44161
I20260812 19:03:27.313294 24328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:44161 every 8 connection(s)
I20260812 19:03:27.314005 24248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:27.316773 24329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:27.317144 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 24248
I20260812 19:03:27.317277 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:27.317588 30147 external_mini_cluster.cc:1428] Setting key 9c93bb4936ffe151c2d78726fb43bb31
I20260812 19:03:27.321475 24329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.112.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.253@KRBTEST.COM" created.
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:03:27.337605 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.323931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44161 (local address 127.0.0.1:54326)
0812 19:03:27.324527 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.324564 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:27.325407 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:27.326144 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:27.326149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:27.326585 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:27.326846 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.326859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.328043 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.328046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.328954 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.328967 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.329707 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.329711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.329844 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.330503 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:27.330522 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:27.332141 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:27.334137 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.334147 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.334163 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.334495 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.334821 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.334828 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.334833 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.335014 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.335903 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:27.335947 (+ 44us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:27.336230 (+ 283us) client_negotiation.cc:769] Sending connection context
0812 19:03:27.336438 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 19:03:27.336871 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":99,"spinlock_wait_cycles":4352,"thread_start_us":156,"threads_started":1}
I20260812 19:03:27.337604 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.323930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54326 (local address 127.29.112.254:44161)
0812 19:03:27.324381 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.324400 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.324682 (+ 282us) server_negotiation.cc:407] Connection header received
0812 19:03:27.325715 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.325718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.325955 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.326153 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.327006 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.327914 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.329079 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.329534 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.329715 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.332307 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.332312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.332322 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.332345 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.334023 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.334605 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.334609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.334610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.334717 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.335146 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.335152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.335154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.335750 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.335909 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.336516 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.336753 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":299,"thread_start_us":157,"threads_started":1}
I20260812 19:03:27.340021 24329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:03:27.340791 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.340539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:33981 (local address 127.0.0.1:54744)
0812 19:03:27.340661 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.340733 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 19:03:27.341050 24270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.253:33981: connect: Connection refused (error 111)
I20260812 19:03:27.342046 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.341717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:33981 (local address 127.0.0.1:54756)
0812 19:03:27.341924 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.341978 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 19:03:27.342276 24329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.253:33981: Network error: Client connection negotiation failed: client connection to 127.29.112.253:33981: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab.
Entry for principal kudu/127.29.112.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab.
I20260812 19:03:27.349381 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.253:33981
--webserver_interface=127.29.112.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab
--principal=kudu/127.29.112.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:44161,127.29.112.253:33981,127.29.112.252:43581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:03:27.391569 24329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } attempt: 1
I20260812 19:03:27.392609 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.392264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:33981 (local address 127.0.0.1:54770)
0812 19:03:27.392452 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.392524 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 19:03:27.393452 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.393176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:33981 (local address 127.0.0.1:54782)
0812 19:03:27.393296 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.393356 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 19:03:27.393584 24329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.253:33981: Network error: Client connection negotiation failed: client connection to 127.29.112.253:33981: connect: Connection refused (error 111)
I20260812 19:03:27.461776 24329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } attempt: 2
I20260812 19:03:27.462745 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.462450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:33981 (local address 127.0.0.1:54796)
0812 19:03:27.462608 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.462671 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 19:03:27.463507 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.463278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:33981 (local address 127.0.0.1:54812)
0812 19:03:27.463394 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.463444 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 19:03:27.463650 24329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.253:33981: Network error: Client connection negotiation failed: client connection to 127.29.112.253:33981: connect: Connection refused (error 111)
W20260812 19:03:27.464610 24337 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:03:27.464927 24337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:27.464991 24337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:27.465067 24337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:27.469089 24337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:27.469178 24337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:27.469228 24337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:27.469285 24337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:27.469321 24337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:27.474784 24337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:44161,127.29.112.253:33981,127.29.112.252:43581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab
--principal=kudu/127.29.112.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.253:33981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.112.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:27.475927 24337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:27.477187 24337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:27.485837 24345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:27.486039 24337 server_base.cc:1034] running on GCE node
W20260812 19:03:27.485837 24343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:27.485970 24342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:27.486730 24337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:27.487874 24337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:27.489104 24337 hybrid_clock.cc:648] HybridClock initialized: now 1786561407489070 us; error 45 us; skew 500 ppm
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:27.492304 24337 init.cc:377] Logged in from keytab as kudu/127.29.112.253@KRBTEST.COM (short username kudu)
I20260812 19:03:27.493285 24337 webserver.cc:460] Webserver started at http://127.29.112.253:43689/ using document root <none> and password file <none>
I20260812 19:03:27.493922 24337 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:27.494020 24337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:27.494269 24337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:27.496420 24337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/data/instance:
uuid: "0db77c94cbc54144b1a33b19e99e752f"
format_stamp: "Formatted at 2026-08-12 19:03:27 on dist-test-slave-6nmv"
server_key: "47c16002617eaabe404ccc1a39e51c43"
server_key_iv: "ffab9ef097cc80eec46aed57222528b1"
server_key_version: "encryptionkey@0"
I20260812 19:03:27.497037 24337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/wal/instance:
uuid: "0db77c94cbc54144b1a33b19e99e752f"
format_stamp: "Formatted at 2026-08-12 19:03:27 on dist-test-slave-6nmv"
server_key: "47c16002617eaabe404ccc1a39e51c43"
server_key_iv: "ffab9ef097cc80eec46aed57222528b1"
server_key_version: "encryptionkey@0"
I20260812 19:03:27.500998 24337 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:03:27.503573 24352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:27.504757 24337 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:03:27.504906 24337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/wal
uuid: "0db77c94cbc54144b1a33b19e99e752f"
format_stamp: "Formatted at 2026-08-12 19:03:27 on dist-test-slave-6nmv"
server_key: "47c16002617eaabe404ccc1a39e51c43"
server_key_iv: "ffab9ef097cc80eec46aed57222528b1"
server_key_version: "encryptionkey@0"
I20260812 19:03:27.505025 24337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:27.526037 24337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:27.529213 24337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:27.529439 24337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:27.536809 24337 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.253:33981
I20260812 19:03:27.537114 24417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.253:33981 every 8 connection(s)
I20260812 19:03:27.537915 24337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
I20260812 19:03:27.540606 24418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:27.544345 24418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:03:27.547132 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 24337
I20260812 19:03:27.547264 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-1/wal/instance
I20260812 19:03:27.547582 30147 external_mini_cluster.cc:1428] Setting key 6deb4a284b5480946a66e63013cf3669
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:03:27.554808 24329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } attempt: 3
WARNING: no policy specified for kudu/127.29.112.252@KRBTEST.COM; defaulting to no policy
I20260812 19:03:27.560106 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.546259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54340 (local address 127.29.112.254:44161)
0812 19:03:27.546421 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.546427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.547369 (+ 942us) server_negotiation.cc:407] Connection header received
0812 19:03:27.548732 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.548735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.548789 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.548868 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.550382 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.551174 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.552274 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.552518 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.552636 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.555231 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.555235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.555240 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.555268 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.557794 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.558415 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.558418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.558420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.558511 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.558837 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.558841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.558842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.559050 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.559164 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.559768 (+ 604us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.559972 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.252@KRBTEST.COM" created.
I20260812 19:03:27.561319 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.546681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44161 (local address 127.0.0.1:54340)
0812 19:03:27.547198 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.547262 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:27.548489 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:27.549078 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:27.549089 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:27.549621 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:27.550230 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.550243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.551288 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.551291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.552146 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.552154 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.552675 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.552679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.552846 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.553473 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:27.553490 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:27.555062 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:27.557938 (+ 2876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.557944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.557958 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.558261 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.558599 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.558601 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.558603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.558743 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.559160 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:27.559165 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:27.559512 (+ 347us) client_negotiation.cc:769] Sending connection context
0812 19:03:27.559673 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 19:03:27.559985 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":123,"threads_started":1}
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
I20260812 19:03:27.563004 24418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:03:27.566581 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.555309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:33981 (local address 127.0.0.1:54816)
0812 19:03:27.555528 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.555544 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:27.555622 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:27.556377 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:27.556383 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:27.556588 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:27.557198 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.557210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.558377 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.558383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.559132 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.559140 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.559858 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.559861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.560035 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.560515 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:27.560530 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:27.562128 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:27.564688 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.564691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.564694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.564947 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.565282 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.565286 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.565288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.565380 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.566072 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:27.566075 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:27.566174 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 19:03:27.566285 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 19:03:27.566420 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260812 19:03:27.567107 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.555399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.29.112.253:33981)
0812 19:03:27.555885 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.555898 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.555924 (+ 26us) server_negotiation.cc:407] Connection header received
0812 19:03:27.555973 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.555979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.556198 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.556355 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.557350 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.558258 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.559262 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.559720 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.559832 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.562291 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.562304 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.562338 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.562369 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.564580 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.565064 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.565073 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.565077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.565176 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.565493 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.565499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.565503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.565922 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.566108 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.566410 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.566569 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":151,"threads_started":1}
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
I20260812 19:03:27.569276 24329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:03:27.570106 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.569846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:33898)
0812 19:03:27.569977 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.570036 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 19:03:27.571029 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.570719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:33908)
0812 19:03:27.570894 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.570950 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260812 19:03:27.571220 24329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:43581: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
I20260812 19:03:27.572237 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.563657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:33981 (local address 127.0.0.1:54830)
0812 19:03:27.563843 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.563857 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:27.563932 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:27.564228 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:27.564233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:27.564527 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:27.564680 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.564690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.565616 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.565621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.566349 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.566359 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.567032 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.567034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.567096 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.567560 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:27.567571 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:27.568729 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:27.570853 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.570859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.570862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.571051 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.571354 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.571357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.571359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.571452 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.571796 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:27.571798 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:27.571930 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 19:03:27.572034 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 19:03:27.572138 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260812 19:03:27.572470 24425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.563676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.29.112.253:33981)
0812 19:03:27.563984 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.563992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.564008 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:03:27.564041 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.564045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.564091 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.564187 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.564801 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.565513 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.566480 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.566839 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.566937 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.568877 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.568882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.568884 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.568912 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.570747 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.571174 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.571177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.571178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.571255 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.571537 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.571539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.571540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.571710 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.571786 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.572133 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.572234 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":88,"threads_started":1}
I20260812 19:03:27.573076 24418 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:03:27.573765 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.573522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:33922)
0812 19:03:27.573645 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.573713 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 19:03:27.574070 24353 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
I20260812 19:03:27.574898 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.574718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:33934)
0812 19:03:27.574809 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.574851 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 19:03:27.575062 24418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:43581: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab.
Entry for principal kudu/127.29.112.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab.
I20260812 19:03:27.580304 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.252:43581
--webserver_interface=127.29.112.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab
--principal=kudu/127.29.112.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:44161,127.29.112.253:33981,127.29.112.252:43581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:03:27.602382 24329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } attempt: 1
I20260812 19:03:27.603268 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.602977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:33938)
0812 19:03:27.603128 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.603190 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 19:03:27.603973 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.603765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:33952)
0812 19:03:27.603869 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.603918 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 19:03:27.604112 24329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:43581: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
I20260812 19:03:27.624481 24418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } attempt: 1
I20260812 19:03:27.625437 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.625137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:33962)
0812 19:03:27.625290 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.625355 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 19:03:27.626164 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.625961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:33976)
0812 19:03:27.626065 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.626115 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 19:03:27.626291 24418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:43581: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
I20260812 19:03:27.679275 24329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } attempt: 2
I20260812 19:03:27.680158 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.679850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:33980)
0812 19:03:27.680005 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.680076 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 19:03:27.680891 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.680668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:33984)
0812 19:03:27.680782 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.680832 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 19:03:27.681030 24329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:43581: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
I20260812 19:03:27.694444 24418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } attempt: 2
W20260812 19:03:27.694680 24427 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:03:27.694988 24427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:27.695050 24427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:27.695122 24427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 19:03:27.695307 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.694996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:33998)
0812 19:03:27.695145 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.695215 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 19:03:27.696027 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.695800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:34000)
0812 19:03:27.695907 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.695953 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 19:03:27.696199 24418 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:43581: Network error: Client connection negotiation failed: client connection to 127.29.112.252:43581: connect: Connection refused (error 111)
W20260812 19:03:27.699385 24427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:27.699441 24427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:27.699461 24427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:27.699478 24427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:27.699494 24427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:27.704759 24427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:44161,127.29.112.253:33981,127.29.112.252:43581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab
--principal=kudu/127.29.112.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.252:43581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.112.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:27.705756 24427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:27.706853 24427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:27.715539 24436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:27.715576 24434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:27.715576 24433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:27.715893 24427 server_base.cc:1034] running on GCE node
I20260812 19:03:27.716539 24427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:27.717653 24427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:27.718847 24427 hybrid_clock.cc:648] HybridClock initialized: now 1786561407718833 us; error 35 us; skew 500 ppm
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:27.727918 24427 init.cc:377] Logged in from keytab as kudu/127.29.112.252@KRBTEST.COM (short username kudu)
I20260812 19:03:27.728881 24427 webserver.cc:460] Webserver started at http://127.29.112.252:46773/ using document root <none> and password file <none>
I20260812 19:03:27.729440 24427 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:27.729501 24427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:27.729665 24427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:27.731776 24427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/data/instance:
uuid: "9b425182ca0a434ea02ce550d25c6237"
format_stamp: "Formatted at 2026-08-12 19:03:27 on dist-test-slave-6nmv"
server_key: "725510ad33f1581e89a3f0ad9ef7dba2"
server_key_iv: "bc10b58e25042c854298129f46891eab"
server_key_version: "encryptionkey@0"
I20260812 19:03:27.732291 24427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/wal/instance:
uuid: "9b425182ca0a434ea02ce550d25c6237"
format_stamp: "Formatted at 2026-08-12 19:03:27 on dist-test-slave-6nmv"
server_key: "725510ad33f1581e89a3f0ad9ef7dba2"
server_key_iv: "bc10b58e25042c854298129f46891eab"
server_key_version: "encryptionkey@0"
I20260812 19:03:27.736227 24427 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:03:27.738731 24444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:27.739953 24427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:03:27.740080 24427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/wal
uuid: "9b425182ca0a434ea02ce550d25c6237"
format_stamp: "Formatted at 2026-08-12 19:03:27 on dist-test-slave-6nmv"
server_key: "725510ad33f1581e89a3f0ad9ef7dba2"
server_key_iv: "bc10b58e25042c854298129f46891eab"
server_key_version: "encryptionkey@0"
I20260812 19:03:27.740193 24427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:27.761376 24427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:27.764564 24427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:27.764765 24427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:27.772120 24427 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.252:43581
I20260812 19:03:27.772425 24507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.252:43581 every 8 connection(s)
I20260812 19:03:27.773119 24427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
I20260812 19:03:27.775914 24508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:27.777467 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 24427
I20260812 19:03:27.777603 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786561125017040-30147-0/minicluster-data/master-2/wal/instance
I20260812 19:03:27.777933 30147 external_mini_cluster.cc:1428] Setting key 587f3a8719db7234a389da87b4ddf188
I20260812 19:03:27.780205 24329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } attempt: 3
I20260812 19:03:27.780174 24508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
I20260812 19:03:27.787402 24418 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } attempt: 3
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:03:27.793695 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.780721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:34028)
0812 19:03:27.780964 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.780982 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:27.781081 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:27.781404 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:27.781407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:27.781623 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:27.781733 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.781741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.783821 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.783827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.784338 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.784345 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.785539 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.785542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.785633 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.786260 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:27.786275 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:27.787721 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:27.790351 (+ 2630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.790354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.790357 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.790630 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.791752 (+ 1122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.791757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.791759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.791864 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.792759 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:27.792762 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:27.792895 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 19:03:27.793071 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 19:03:27.793480 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"spinlock_wait_cycles":1792}
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
I20260812 19:03:27.794574 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.783358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.29.112.254:44161)
0812 19:03:27.783492 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.783498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.783514 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:03:27.783551 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.783557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.783607 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.783691 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.784476 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.785016 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.786080 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.786307 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.786416 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.790516 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.790520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.790523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.790548 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.792300 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.792856 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.792859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.792861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.792945 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.793303 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.793308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.793311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.793535 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.793605 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.794327 (+ 722us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.794437 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60,"spinlock_wait_cycles":1792}
I20260812 19:03:27.795192 24513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.780766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.29.112.252:43581)
0812 19:03:27.781144 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.781156 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.781172 (+ 16us) server_negotiation.cc:407] Connection header received
0812 19:03:27.781210 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.781214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.781254 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.781482 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.781984 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.783491 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.784457 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.785314 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.785657 (+ 343us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.787875 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.787882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.787894 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.787918 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.790241 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.791125 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.791132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.791139 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.791292 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.791978 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.791982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.791983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.792570 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.792709 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.794052 (+ 1343us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.794165 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":234,"server-negotiator.queue_time_us":306,"thread_start_us":158,"threads_started":1}
I20260812 19:03:27.795666 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.782577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:44161 (local address 127.0.0.1:54352)
0812 19:03:27.782922 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.782972 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:27.783076 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:27.783734 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:27.783740 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:27.784195 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:27.784330 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.784337 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.785129 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.785131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.785950 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.785960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.786457 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.786460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.786579 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.787121 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:27.787148 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:27.790395 (+ 3247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:27.792420 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.792427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.792433 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.792752 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.793080 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.793083 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.793085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.793180 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.793687 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:27.793693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:27.794098 (+ 405us) client_negotiation.cc:769] Sending connection context
0812 19:03:27.794930 (+ 832us) client_negotiation.cc:240] Negotiation successful
0812 19:03:27.795118 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":85,"threads_started":1}
I20260812 19:03:27.795192 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.779317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34012 (local address 127.29.112.252:43581)
0812 19:03:27.779716 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.779741 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.779775 (+ 34us) server_negotiation.cc:407] Connection header received
0812 19:03:27.780502 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.780525 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.781105 (+ 580us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.782243 (+ 1138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.782512 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.783495 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.784634 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.785314 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.785518 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.789525 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.789532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.789536 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.789588 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.791560 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.792073 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.792076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.792078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.792163 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.792459 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.792462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.792464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.792702 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.792877 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.793748 (+ 871us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.794100 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":281,"server-negotiator.queue_time_us":256,"spinlock_wait_cycles":4864,"thread_start_us":127,"threads_started":1}
I20260812 19:03:27.797029 24508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:03:27.800320 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.787852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:34030)
0812 19:03:27.788123 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.788139 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:27.788513 (+ 374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:27.788815 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:27.788818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:27.789228 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:27.789317 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.789324 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.791398 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.791402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.792273 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.792284 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.792859 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.792863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.792948 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.793430 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:27.793447 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:27.794889 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:27.798662 (+ 3773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.798669 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.798674 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.798924 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.799257 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.799260 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.799262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.799364 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.799798 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:27.799801 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:27.799904 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 19:03:27.800021 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 19:03:27.800169 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260812 19:03:27.800336 24516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.788113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34030 (local address 127.29.112.252:43581)
0812 19:03:27.788394 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.788403 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.788418 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:03:27.788652 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.788655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.788702 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.788795 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.789464 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.791250 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.792416 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.792729 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.793176 (+ 447us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.795038 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.795042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.795044 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.795067 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.798527 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.799047 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.799049 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.799051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.799147 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.799471 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.799476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.799481 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.799691 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.799794 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.800070 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.800173 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":100,"threads_started":1}
I20260812 19:03:27.800978 24329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c: Bootstrap starting.
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
I20260812 19:03:27.804024 24329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:27.805003 24329 log.cc:824] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:27.805764 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.797562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:33981 (local address 127.0.0.1:54832)
0812 19:03:27.797660 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.797676 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:27.797747 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:27.798033 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:27.798036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:27.798332 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:27.798427 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.798432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.799180 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.799187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.799859 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.799869 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.800531 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.800535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.800630 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.801025 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:27.801039 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:27.802386 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:27.804242 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.804244 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.804246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.804432 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.804718 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.804720 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.804721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.804792 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.805293 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:27.805295 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:27.805372 (+ 77us) client_negotiation.cc:769] Sending connection context
0812 19:03:27.805484 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 19:03:27.805614 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 19:03:27.805886 24425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.797649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54832 (local address 127.29.112.253:33981)
0812 19:03:27.797773 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.797777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.797790 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:03:27.797853 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.797856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.797909 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.798000 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.798553 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.799071 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.799987 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.800315 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.800417 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.802521 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.802525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.802528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.802553 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.804114 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.804527 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.804532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.804534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.804625 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.804884 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.804888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.804890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.805193 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.805296 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.805558 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.805681 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 19:03:27.806818 24508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:03:27.807435 24329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c: No bootstrap required, opened a new log
I20260812 19:03:27.808686 24418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f: Bootstrap starting.
I20260812 19:03:27.811460 24418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f: Neither blocks nor log segments found. Creating new log.
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
I20260812 19:03:27.811466 24329 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } }
I20260812 19:03:27.811759 24329 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad24d36f6b3a4b63932f986bf9cb1b3c, State: Initialized, Role: FOLLOWER
I20260812 19:03:27.812109 24329 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } }
I20260812 19:03:27.812363 24418 log.cc:824] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:27.813599 24329 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:27.813059 24519 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } } }
I20260812 19:03:27.814019 24519 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:27.815470 24516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.807482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34046 (local address 127.29.112.252:43581)
0812 19:03:27.807579 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.807583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.807704 (+ 121us) server_negotiation.cc:407] Connection header received
0812 19:03:27.807812 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.807816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.807851 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.807939 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.808285 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.808846 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.809631 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.809957 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.810063 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.812087 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.812091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.812093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.812115 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.813705 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.814338 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.814341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.814343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.814418 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.814676 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.814678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.814680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.814849 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.814906 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.815204 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.815319 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 19:03:27.815711 24418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f: No bootstrap required, opened a new log
I20260812 19:03:27.816064 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.807363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:43581 (local address 127.0.0.1:34046)
0812 19:03:27.807635 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:27.807645 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:27.807742 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:27.807950 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:27.807952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:27.808118 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:27.808180 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.808185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.808935 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.808941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.809523 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:27.809532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.810069 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.810072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:27.810147 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.810636 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:27.810650 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:27.811958 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:27.813842 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.813848 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.813852 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.814239 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.814520 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:27.814523 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:27.814525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:27.814597 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:27.814960 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:27.814962 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:27.815045 (+ 83us) client_negotiation.cc:769] Sending connection context
0812 19:03:27.815805 (+ 760us) client_negotiation.cc:240] Negotiation successful
0812 19:03:27.815929 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230}
W20260812 19:03:27.819447 24534 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 19:03:27.819466 24508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237: Bootstrap starting.
W20260812 19:03:27.819532 24534 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:27.819727 24418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } }
I20260812 19:03:27.819957 24418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0db77c94cbc54144b1a33b19e99e752f, State: Initialized, Role: FOLLOWER
I20260812 19:03:27.820276 24418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } }
I20260812 19:03:27.820785 24536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } } }
I20260812 19:03:27.820971 24536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:27.821630 24418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:27.821928 24508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237: Neither blocks nor log segments found. Creating new log.
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
W20260812 19:03:27.827476 24550 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:03:27.827567 24550 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:27.823149 24248 master_runner.cc:186] Error getting master registration for 127.29.112.253:33981: OK, instance_id { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" instance_seqno: 1786561407534829 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 19:03:27.828364 24248 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:03:27.829205 24508 log.cc:824] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:27.829919 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.798025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54364 (local address 127.29.112.254:44161)
0812 19:03:27.798121 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.798124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.812906 (+ 14782us) server_negotiation.cc:407] Connection header received
0812 19:03:27.813127 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.813131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.813180 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.813233 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.814208 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.814953 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.816204 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.816549 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.816608 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.825616 (+ 9008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.825621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.825623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.825653 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.827448 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.828087 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.828091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.828093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.828184 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.828680 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.828683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.828685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.828921 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.829095 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.829579 (+ 484us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.829724 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 19:03:27.831015 24337 master_runner.cc:186] Error getting master registration for 127.29.112.252:43581: OK, instance_id { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" instance_seqno: 1786561407770083 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 19:03:27.834481 24337 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 19:03:27 dist-test-slave-6nmv krb5kdc[24234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
I20260812 19:03:27.839996 24425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.831475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54836 (local address 127.29.112.253:33981)
0812 19:03:27.831585 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.831589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.831711 (+ 122us) server_negotiation.cc:407] Connection header received
0812 19:03:27.831845 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.831848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.831888 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.831980 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.832603 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.833109 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.833923 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.834165 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.834279 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.836796 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.836800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.836802 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.836829 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.838337 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.838763 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.838767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.838769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.838853 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.839142 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.839146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.839148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.839335 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.839574 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.839695 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.839815 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 19:03:27.843526 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:27.844911 24299 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54364
I20260812 19:03:27.845172 24299 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:03:27.845291 24299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:27.846047 24508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237: No bootstrap required, opened a new log
I20260812 19:03:27.846771 24299 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } }
I20260812 19:03:27.847710 24299 leader_election.cc:288] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [CANDIDATE]: Term 1 election: Requested vote from peers 0db77c94cbc54144b1a33b19e99e752f (127.29.112.253:33981), 9b425182ca0a434ea02ce550d25c6237 (127.29.112.252:43581)
I20260812 19:03:27.848156 24388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0db77c94cbc54144b1a33b19e99e752f"
I20260812 19:03:27.848418 24388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:27.849762 24388 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad24d36f6b3a4b63932f986bf9cb1b3c in term 1.
I20260812 19:03:27.850238 24270 leader_election.cc:302] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0db77c94cbc54144b1a33b19e99e752f, ad24d36f6b3a4b63932f986bf9cb1b3c; no voters:
I20260812 19:03:27.850548 24519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:27.851069 24519 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [term 1 LEADER]: Becoming Leader. State: Replica: ad24d36f6b3a4b63932f986bf9cb1b3c, State: Running, Role: LEADER
I20260812 19:03:27.851821 24519 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } }
I20260812 19:03:27.855278 24519 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad24d36f6b3a4b63932f986bf9cb1b3c. Latest consensus state: current_term: 1 leader_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } } }
I20260812 19:03:27.855561 24519 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:27.855665 24508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } }
I20260812 19:03:27.855940 24508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b425182ca0a434ea02ce550d25c6237, State: Initialized, Role: FOLLOWER
I20260812 19:03:27.856309 24508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } }
I20260812 19:03:27.856860 24563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } } }
I20260812 19:03:27.857120 24563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:27.858738 24508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:27.858783 24478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9b425182ca0a434ea02ce550d25c6237"
I20260812 19:03:27.858997 24478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:27.860567 24478 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad24d36f6b3a4b63932f986bf9cb1b3c in term 1.
I20260812 19:03:27.867574 24562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:27.868538 24562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260812 19:03:27.880173 24578 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:03:27.880275 24578 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:27.890524 24478 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [term 1 FOLLOWER]: Refusing update from remote peer ad24d36f6b3a4b63932f986bf9cb1b3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:27.891408 24388 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [term 1 FOLLOWER]: Refusing update from remote peer ad24d36f6b3a4b63932f986bf9cb1b3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:27.892264 24425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.858361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54840 (local address 127.29.112.253:33981)
0812 19:03:27.858511 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.858516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.862189 (+ 3673us) server_negotiation.cc:407] Connection header received
0812 19:03:27.862420 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.862424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.862481 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.862538 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.863895 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.864657 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.865777 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.866122 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.866201 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.883752 (+ 17551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.883757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.883760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.883792 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.885553 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.886107 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.886112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.886114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.886202 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.888830 (+ 2628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.888834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.888836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.889115 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.889236 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.891980 (+ 2744us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.892113 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 19:03:27.892354 24554 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:27.894596 24554 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:27.895375 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.861371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54366 (local address 127.29.112.254:44161)
0812 19:03:27.861513 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.861518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.870211 (+ 8693us) server_negotiation.cc:407] Connection header received
0812 19:03:27.870393 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.870396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.870447 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.870502 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.871534 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.872316 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.873510 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.873869 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.873948 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.886551 (+ 12603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.886555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.886558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.886588 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.888316 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.892752 (+ 4436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.892755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.892757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.892860 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.893294 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.893300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.893303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.893557 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.893697 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.895063 (+ 1366us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.895217 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 19:03:27.896468 24516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:27.861974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34062 (local address 127.29.112.252:43581)
0812 19:03:27.862293 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:27.862298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:27.874896 (+ 12598us) server_negotiation.cc:407] Connection header received
0812 19:03:27.875094 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:27.875097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:27.875147 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:27.875197 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:27.876226 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.877147 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.880418 (+ 3271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:27.880758 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:27.880835 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:27.889606 (+ 8771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:27.889610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:27.889613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:27.889642 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:27.891379 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.894105 (+ 2726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.894109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.894111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.894213 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:27.894556 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:27.894559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:27.894562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:27.894793 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:27.894914 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:27.896209 (+ 1295us) server_negotiation.cc:299] Negotiation successful
0812 19:03:27.896332 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248}
I20260812 19:03:27.903550 24563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad24d36f6b3a4b63932f986bf9cb1b3c. Latest consensus state: current_term: 1 leader_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } } }
I20260812 19:03:27.903689 24563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:27.904284 24536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad24d36f6b3a4b63932f986bf9cb1b3c. Latest consensus state: current_term: 1 leader_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } } }
I20260812 19:03:27.904398 24536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:27.914821 24554 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } } }
I20260812 19:03:27.914964 24554 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:27.915087 24582 mvcc.cc:205] Tried to move back new op lower bound from 7317755526671855616 to 7317755526567825408. Current Snapshot: MvccSnapshot[applied={T|T < 7317755526671855616}]
I20260812 19:03:27.916491 24584 mvcc.cc:205] Tried to move back new op lower bound from 7317755526671855616 to 7317755526567825408. Current Snapshot: MvccSnapshot[applied={T|T < 7317755526671855616}]
I20260812 19:03:27.917315 24519 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } } }
I20260812 19:03:27.917426 24519 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:27.917640 24536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } } }
I20260812 19:03:27.917745 24536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:27.921120 24563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad24d36f6b3a4b63932f986bf9cb1b3c" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 44161 } } peers { permanent_uuid: "0db77c94cbc54144b1a33b19e99e752f" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 33981 } } peers { permanent_uuid: "9b425182ca0a434ea02ce550d25c6237" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 43581 } } }
I20260812 19:03:27.921227 24563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:27.921856 24585 mvcc.cc:205] Tried to move back new op lower bound from 7317755526671855616 to 7317755526567825408. Current Snapshot: MvccSnapshot[applied={T|T < 7317755526671855616}]
I20260812 19:03:27.927124 24562 catalog_manager.cc:1347] Generated new cluster ID: 6620d1c8cda2431cb73e121e308b7411
I20260812 19:03:27.927199 24562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:27.964315 24562 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:27.966209 24562 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:27.985883 24562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad24d36f6b3a4b63932f986bf9cb1b3c: Generated new TSK 0
I20260812 19:03:27.986709 24562 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 19:03:28.025493 24425 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.022087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54844 (local address 127.29.112.253:33981)
0812 19:03:28.022271 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.022278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.024389 (+ 2111us) server_negotiation.cc:407] Connection header received
0812 19:03:28.024583 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.024587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.024652 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.024708 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.025358 (+ 650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54844: BlockingRecv error: recv got EOF from 127.0.0.1:54844 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 19:03:28.026996 24333 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.023152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54368 (local address 127.29.112.254:44161)
0812 19:03:28.023263 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.023267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.026045 (+ 2778us) server_negotiation.cc:407] Connection header received
0812 19:03:28.026238 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.026242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.026289 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.026340 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.026878 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54368: BlockingRecv error: recv got EOF from 127.0.0.1:54368 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 19:03:28.028192 24516 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.023856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.29.112.252:43581)
0812 19:03:28.024012 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.024016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.027376 (+ 3360us) server_negotiation.cc:407] Connection header received
0812 19:03:28.027472 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.027476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.027534 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.027632 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.028083 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34076: BlockingRecv error: recv got EOF from 127.0.0.1:34076 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 19:03:28.032208 24425 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.030674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54848 (local address 127.29.112.253:33981)
0812 19:03:28.030776 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.030779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.031221 (+ 442us) server_negotiation.cc:407] Connection header received
0812 19:03:28.031420 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.031423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.031491 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.031539 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.032079 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54848: BlockingRecv error: recv got EOF from 127.0.0.1:54848 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 19:03:28.037626 24425 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.035954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.29.112.253:33981)
0812 19:03:28.036057 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.036061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.036566 (+ 505us) server_negotiation.cc:407] Connection header received
0812 19:03:28.036798 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.036803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.036868 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.036929 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.037506 (+ 577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54852: BlockingRecv error: recv got EOF from 127.0.0.1:54852 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 19:03:28.043094 24425 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.041525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54866 (local address 127.29.112.253:33981)
0812 19:03:28.041651 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.041658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.042214 (+ 556us) server_negotiation.cc:407] Connection header received
0812 19:03:28.042383 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.042386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.042432 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.042480 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.042995 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54866: BlockingRecv error: recv got EOF from 127.0.0.1:54866 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 19:03:28.050235 24425 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.049006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54878 (local address 127.29.112.253:33981)
0812 19:03:28.049162 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.049166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.049366 (+ 200us) server_negotiation.cc:407] Connection header received
0812 19:03:28.049491 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.049496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.049556 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.049663 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.050134 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54878: BlockingRecv error: recv got EOF from 127.0.0.1:54878 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 19:03:28.061525 24425 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.060216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54882 (local address 127.29.112.253:33981)
0812 19:03:28.060338 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.060342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.060666 (+ 324us) server_negotiation.cc:407] Connection header received
0812 19:03:28.060826 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.060829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.060875 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.060966 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.061408 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54882: BlockingRecv error: recv got EOF from 127.0.0.1:54882 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 19:03:28.082042 24425 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.080058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.29.112.253:33981)
0812 19:03:28.080231 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.080238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.080438 (+ 200us) server_negotiation.cc:407] Connection header received
0812 19:03:28.080783 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.080788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.080893 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.080991 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.081895 (+ 904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54892: BlockingRecv error: recv got EOF from 127.0.0.1:54892 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260812 19:03:28.117664 24425 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.116370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54906 (local address 127.29.112.253:33981)
0812 19:03:28.116555 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.116559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.116711 (+ 152us) server_negotiation.cc:407] Connection header received
0812 19:03:28.116894 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.116897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.116954 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.117045 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.117540 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54906: BlockingRecv error: recv got EOF from 127.0.0.1:54906 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260812 19:03:28.184983 24425 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.183887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54918 (local address 127.29.112.253:33981)
0812 19:03:28.184166 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.184169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.184186 (+ 17us) server_negotiation.cc:407] Connection header received
0812 19:03:28.184248 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.184251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.184299 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.184373 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.184838 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54918: BlockingRecv error: recv got EOF from 127.0.0.1:54918 (error 108)
Metrics: {"server-negotiator.queue_time_us":180}
W20260812 19:03:28.316551 24425 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.315247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54924 (local address 127.29.112.253:33981)
0812 19:03:28.315399 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.315405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.315635 (+ 230us) server_negotiation.cc:407] Connection header received
0812 19:03:28.315770 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.315772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.315821 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.315894 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.316408 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54924: BlockingRecv error: recv got EOF from 127.0.0.1:54924 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 19:03:28.576140 24425 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:28.574910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54936 (local address 127.29.112.253:33981)
0812 19:03:28.575101 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:28.575105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:28.575218 (+ 113us) server_negotiation.cc:407] Connection header received
0812 19:03:28.575350 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:28.575352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:28.575404 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:28.575483 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:28.576010 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54936: BlockingRecv error: recv got EOF from 127.0.0.1:54936 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 19:03:28.828966 24550 catalog_manager.cc:1259] Loaded cluster ID: 6620d1c8cda2431cb73e121e308b7411
I20260812 19:03:28.829062 24550 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f: loading cluster ID for follower catalog manager: success
I20260812 19:03:28.830747 24550 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f: acquiring CA information for follower catalog manager: success
I20260812 19:03:28.831882 24550 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0db77c94cbc54144b1a33b19e99e752f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:03:28.881644 24578 catalog_manager.cc:1259] Loaded cluster ID: 6620d1c8cda2431cb73e121e308b7411
I20260812 19:03:28.881735 24578 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237: loading cluster ID for follower catalog manager: success
I20260812 19:03:28.883553 24578 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237: acquiring CA information for follower catalog manager: success
I20260812 19:03:28.884742 24578 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9b425182ca0a434ea02ce550d25c6237: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:03:29.094470 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:29.090283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54942 (local address 127.29.112.253:33981)
0812 19:03:29.090553 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:29.090558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:29.090653 (+ 95us) server_negotiation.cc:407] Connection header received
0812 19:03:29.090847 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:29.090851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:29.090991 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:29.091114 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:29.091397 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:29.091961 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:29.092890 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:29.093123 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:29.093232 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:29.093580 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:29.094080 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:29.094218 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:29.094254 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 19:03:29.094309 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260812 19:03:29.101380 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:29.097639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34080 (local address 127.29.112.252:43581)
0812 19:03:29.097913 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:29.097917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:29.097931 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:03:29.098103 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:29.098105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:29.098220 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:29.098345 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:29.098563 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:29.099077 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:29.099909 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:29.100147 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:29.100260 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:29.100562 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:29.100960 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:29.101094 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:29.101156 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 19:03:29.101226 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":68,"threads_started":1}
I20260812 19:03:29.108654 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:29.104856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.29.112.252:43581)
0812 19:03:29.104987 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:29.104991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:29.105003 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:03:29.105044 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:29.105046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:29.105080 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:29.105164 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:29.105512 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:29.106161 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:29.107471 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:29.107718 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:29.107797 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:29.108111 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:29.108184 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:29.108272 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:29.108428 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 19:03:29.108489 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 19:03:29.108822 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:29.104299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.29.112.253:33981)
0812 19:03:29.104444 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:29.104448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:29.104800 (+ 352us) server_negotiation.cc:407] Connection header received
0812 19:03:29.105360 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:29.105364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:29.105425 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:29.105496 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:29.105869 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:29.106540 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:29.107544 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:29.107779 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:29.107887 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:29.108281 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:29.108364 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:29.108407 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:29.108645 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 19:03:29.108704 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 19:03:29.109615 24680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:29.104263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54384 (local address 127.29.112.254:44161)
0812 19:03:29.104518 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:29.104522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:29.104621 (+ 99us) server_negotiation.cc:407] Connection header received
0812 19:03:29.104825 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:29.104829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:29.105009 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:29.105086 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:29.105477 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:29.106251 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:29.107886 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:29.108276 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:29.108366 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:29.108734 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:29.109138 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:29.109235 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:29.109446 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 19:03:29.109505 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260812 19:03:29.112794 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 24248
I20260812 19:03:29.124348 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 24337
I20260812 19:03:29.132243 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 24427
2026-08-12T19:03:29Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3261 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 19:03:29 dist-test-slave-6nmv krb5kdc[24688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 19:03:29 dist-test-slave-6nmv krb5kdc[24688](info): set up 2 sockets
Aug 12 19:03:29 dist-test-slave-6nmv krb5kdc[24688](info): commencing operation
krb5kdc: starting...
W20260812 19:03:30.267843 30147 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/test-user.keytab.
Aug 12 19:03:30 dist-test-slave-6nmv krb5kdc[24688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T19:03:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T19:03:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
Entry for principal kudu/127.29.112.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab.
I20260812 19:03:30.398376 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:42381
--webserver_interface=127.29.112.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:42381,127.29.112.253:42521,127.29.112.252:42715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 19:03:30.510290 24704 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:03:30.510644 24704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:30.510713 24704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:30.510788 24704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 19:03:30.514796 24704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:30.514890 24704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:30.514930 24704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:30.514968 24704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:30.515012 24704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:30.520262 24704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:42381,127.29.112.253:42521,127.29.112.252:42715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.254.keytab
--principal=kudu/127.29.112.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.254:42381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:30.521404 24704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:30.522526 24704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:03:30.530969 24704 server_base.cc:1034] running on GCE node
W20260812 19:03:30.530985 24709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:30.530916 24711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:30.530908 24713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:30.531694 24704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:30.532826 24704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:30.534011 24704 hybrid_clock.cc:648] HybridClock initialized: now 1786561410533995 us; error 31 us; skew 500 ppm
Aug 12 19:03:30 dist-test-slave-6nmv krb5kdc[24688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:30.536787 24704 init.cc:377] Logged in from keytab as kudu/127.29.112.254@KRBTEST.COM (short username kudu)
I20260812 19:03:30.537659 24704 webserver.cc:460] Webserver started at http://127.29.112.254:45159/ using document root <none> and password file <none>
I20260812 19:03:30.538272 24704 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:30.538367 24704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:30.538602 24704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:30.540644 24704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/data/instance:
uuid: "f19b6157d3cb442a83e81700c34260af"
format_stamp: "Formatted at 2026-08-12 19:03:30 on dist-test-slave-6nmv"
server_key: "2e5734002ece34742a293dc146aae1cf"
server_key_iv: "772f251ea463d430fa03bc12585d2b49"
server_key_version: "encryptionkey@0"
I20260812 19:03:30.541224 24704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/wal/instance:
uuid: "f19b6157d3cb442a83e81700c34260af"
format_stamp: "Formatted at 2026-08-12 19:03:30 on dist-test-slave-6nmv"
server_key: "2e5734002ece34742a293dc146aae1cf"
server_key_iv: "772f251ea463d430fa03bc12585d2b49"
server_key_version: "encryptionkey@0"
I20260812 19:03:30.544998 24704 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 19:03:30.547441 24721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:30.548657 24704 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:03:30.548796 24704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/wal
uuid: "f19b6157d3cb442a83e81700c34260af"
format_stamp: "Formatted at 2026-08-12 19:03:30 on dist-test-slave-6nmv"
server_key: "2e5734002ece34742a293dc146aae1cf"
server_key_iv: "772f251ea463d430fa03bc12585d2b49"
server_key_version: "encryptionkey@0"
I20260812 19:03:30.548904 24704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:30.570335 24704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:30.573535 24704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:30.573773 24704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:30.580905 24704 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.254:42381
I20260812 19:03:30.581212 24781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.254:42381 every 8 connection(s)
I20260812 19:03:30.581938 24704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/data/info.pb
I20260812 19:03:30.584865 24782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:30.585250 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 24704
I20260812 19:03:30.585417 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-0/wal/instance
I20260812 19:03:30.585711 30147 external_mini_cluster.cc:1428] Setting key 047d1e2a04e41e5e000317eb6c80cbe5
I20260812 19:03:30.588611 24782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.112.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.253@KRBTEST.COM" created.
Aug 12 19:03:30 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:03:30.606348 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.591583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50270 (local address 127.29.112.254:42381)
0812 19:03:30.591856 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:30.591866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:30.591908 (+ 42us) server_negotiation.cc:407] Connection header received
0812 19:03:30.593050 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:30.593054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:30.593241 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:30.593403 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:30.594590 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.595535 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.596547 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.597073 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.597202 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:30.599773 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:30.599783 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:30.599799 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:30.599831 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:30.602214 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:30.602871 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:30.602878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:30.602880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:30.603036 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:30.603473 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:30.603479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:30.603481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:30.604083 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:30.604225 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:30.604867 (+ 642us) server_negotiation.cc:299] Negotiation successful
0812 19:03:30.605234 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":172,"thread_start_us":109,"threads_started":1}
I20260812 19:03:30.606357 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.590871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:42381 (local address 127.0.0.1:50270)
0812 19:03:30.591435 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.591501 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:30.592701 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:30.593454 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:30.593464 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:30.594008 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:30.594425 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:30.594444 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.595683 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.595686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:30.596423 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:30.596432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.597218 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.597221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:30.597376 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:30.598102 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:30.598119 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:30.599630 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:30.602358 (+ 2728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:30.602369 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:30.602380 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:30.602738 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:30.603158 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:30.603163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:30.603166 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:30.603347 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:30.604217 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:30.604260 (+ 43us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:30.604484 (+ 224us) client_negotiation.cc:769] Sending connection context
0812 19:03:30.604708 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 19:03:30.605133 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":109,"threads_started":1}
I20260812 19:03:30.609616 24782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:03:30.610447 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.610194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:42521 (local address 127.0.0.1:34960)
0812 19:03:30.610315 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.610386 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:42521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 19:03:30.610692 24723 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.253:42521: connect: Connection refused (error 111)
I20260812 19:03:30.611524 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.611310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:42521 (local address 127.0.0.1:34968)
0812 19:03:30.611416 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.611463 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:42521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 19:03:30.611706 24782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.253:42521: Network error: Client connection negotiation failed: client connection to 127.29.112.253:42521: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab.
Entry for principal kudu/127.29.112.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab.
I20260812 19:03:30.616559 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.253:42521
--webserver_interface=127.29.112.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab
--principal=kudu/127.29.112.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:42381,127.29.112.253:42521,127.29.112.252:42715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:03:30.660934 24782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } attempt: 1
I20260812 19:03:30.661918 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.661557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:42521 (local address 127.0.0.1:34976)
0812 19:03:30.661715 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.661783 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:42521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 19:03:30.662743 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.662513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:42521 (local address 127.0.0.1:34982)
0812 19:03:30.662629 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.662685 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:42521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 19:03:30.662880 24782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.253:42521: Network error: Client connection negotiation failed: client connection to 127.29.112.253:42521: connect: Connection refused (error 111)
W20260812 19:03:30.730190 24790 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:03:30.730520 24790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:30.730590 24790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:30.730659 24790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 19:03:30.731034 24782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } attempt: 2
I20260812 19:03:30.731892 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.731568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:42521 (local address 127.0.0.1:34990)
0812 19:03:30.731744 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.731815 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:42521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 19:03:30.732712 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.732493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:42521 (local address 127.0.0.1:34998)
0812 19:03:30.732610 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.732661 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.253:42521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 19:03:30.732829 24782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.253:42521: Network error: Client connection negotiation failed: client connection to 127.29.112.253:42521: connect: Connection refused (error 111)
W20260812 19:03:30.735131 24790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:30.735213 24790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:30.735267 24790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:30.735324 24790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:30.735361 24790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:30.740761 24790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:42381,127.29.112.253:42521,127.29.112.252:42715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.253.keytab
--principal=kudu/127.29.112.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.253:42521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.112.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:30.741901 24790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:30.742933 24790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:03:30.751587 24790 server_base.cc:1034] running on GCE node
W20260812 19:03:30.751643 24798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:30.751812 24795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:30.751811 24796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:30.752404 24790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:30.753463 24790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:30.754627 24790 hybrid_clock.cc:648] HybridClock initialized: now 1786561410754615 us; error 26 us; skew 500 ppm
Aug 12 19:03:30 dist-test-slave-6nmv krb5kdc[24688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:30.757498 24790 init.cc:377] Logged in from keytab as kudu/127.29.112.253@KRBTEST.COM (short username kudu)
I20260812 19:03:30.758457 24790 webserver.cc:460] Webserver started at http://127.29.112.253:41931/ using document root <none> and password file <none>
I20260812 19:03:30.758983 24790 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:30.759042 24790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:30.759203 24790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:30.761191 24790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/data/instance:
uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3"
format_stamp: "Formatted at 2026-08-12 19:03:30 on dist-test-slave-6nmv"
server_key: "66ffb729eafc5da37317387ee785c751"
server_key_iv: "66e6f582b49f838f5224fe5127f9a430"
server_key_version: "encryptionkey@0"
I20260812 19:03:30.761705 24790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/wal/instance:
uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3"
format_stamp: "Formatted at 2026-08-12 19:03:30 on dist-test-slave-6nmv"
server_key: "66ffb729eafc5da37317387ee785c751"
server_key_iv: "66e6f582b49f838f5224fe5127f9a430"
server_key_version: "encryptionkey@0"
I20260812 19:03:30.765360 24790 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 19:03:30.767764 24805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:30.768965 24790 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 19:03:30.769122 24790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/wal
uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3"
format_stamp: "Formatted at 2026-08-12 19:03:30 on dist-test-slave-6nmv"
server_key: "66ffb729eafc5da37317387ee785c751"
server_key_iv: "66e6f582b49f838f5224fe5127f9a430"
server_key_version: "encryptionkey@0"
I20260812 19:03:30.769223 24790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:30.785570 24790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:30.789115 24790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:30.789325 24790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:30.796391 24790 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.253:42521
I20260812 19:03:30.796710 24866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.253:42521 every 8 connection(s)
I20260812 19:03:30.797389 24790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/data/info.pb
I20260812 19:03:30.800185 24867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:30.803668 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 24790
I20260812 19:03:30.803793 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-1/wal/instance
I20260812 19:03:30.804123 30147 external_mini_cluster.cc:1428] Setting key 4cd59d03c0d67789593d1254cdafed7b
I20260812 19:03:30.804131 24867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.112.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.252@KRBTEST.COM" created.
Aug 12 19:03:30 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:03:30.822461 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.806323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50278 (local address 127.29.112.254:42381)
0812 19:03:30.806476 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:30.806480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:30.807358 (+ 878us) server_negotiation.cc:407] Connection header received
0812 19:03:30.808700 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:30.808702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:30.808755 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:30.808835 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:30.810585 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.811134 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.812954 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.813203 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.813280 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:30.818218 (+ 4938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:30.818225 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:30.818231 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:30.818273 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:30.819920 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:30.820618 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:30.820620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:30.820621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:30.820700 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:30.821114 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:30.821117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:30.821118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:30.821317 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:30.821397 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:30.822154 (+ 757us) server_negotiation.cc:299] Negotiation successful
0812 19:03:30.822307 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:03:30.823707 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.806716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:42381 (local address 127.0.0.1:50278)
0812 19:03:30.807181 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.807233 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:30.808473 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:30.809032 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:30.809043 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:30.809618 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:30.810389 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:30.810414 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.811281 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.811288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:30.812813 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:30.812826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.813377 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.813383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:30.813641 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:30.814371 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:30.814401 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:30.818007 (+ 3606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:30.820050 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:30.820059 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:30.820078 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:30.820500 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:30.820793 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:30.820800 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:30.820805 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:30.821005 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:30.821430 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:30.821440 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:30.821800 (+ 360us) client_negotiation.cc:769] Sending connection context
0812 19:03:30.822082 (+ 282us) client_negotiation.cc:240] Negotiation successful
0812 19:03:30.822487 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":135,"threads_started":1}
I20260812 19:03:30.823977 24782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } attempt: 3
I20260812 19:03:30.825332 24867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:03:30 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
Aug 12 19:03:30 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab.
Entry for principal kudu/127.29.112.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab.
I20260812 19:03:30.835994 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.825851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:42521 (local address 127.0.0.1:35004)
0812 19:03:30.826006 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.826019 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:30.826101 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:30.826473 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:30.826478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:30.826662 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:30.826761 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:30.826767 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.827884 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.827887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:30.828615 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:30.828623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.829152 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.829156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:30.829306 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:30.829931 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:30.829949 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:30.831261 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:30.834260 (+ 2999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:30.834265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:30.834268 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:30.834559 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:30.834947 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:30.834949 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:30.834951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:30.835087 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:30.835539 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:30.835541 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:30.835634 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 19:03:30.835743 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 19:03:30.835871 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":43}
I20260812 19:03:30.836133 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.824419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:42521 (local address 127.0.0.1:35000)
0812 19:03:30.824589 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.824606 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:30.824691 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:30.825573 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:30.825576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:30.825776 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:30.826411 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:30.826422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.827566 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.827571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:30.828360 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:30.828371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.829152 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.829156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:30.829264 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:30.829853 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:30.829874 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:30.831891 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:30.834305 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:30.834308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:30.834309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:30.834550 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:30.834860 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:30.834866 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:30.834871 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:30.834982 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:30.835517 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:30.835520 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:30.835626 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 19:03:30.835732 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 19:03:30.835890 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 19:03:30.836642 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.825937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35004 (local address 127.29.112.253:42521)
0812 19:03:30.826204 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:30.826215 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:30.826230 (+ 15us) server_negotiation.cc:407] Connection header received
0812 19:03:30.826268 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:30.826274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:30.826322 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:30.826411 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:30.826897 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.827774 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.828715 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.829025 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.829217 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:30.831446 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:30.831457 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:30.831474 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:30.831503 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:30.834164 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:30.834682 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:30.834691 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:30.834692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:30.834830 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:30.835199 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:30.835202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:30.835203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:30.835415 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:30.835546 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:30.836384 (+ 838us) server_negotiation.cc:299] Negotiation successful
0812 19:03:30.836524 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":203,"spinlock_wait_cycles":4992,"thread_start_us":113,"threads_started":1}
I20260812 19:03:30.836769 30147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
/tmp/dist-test-taskmcQeqc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.252:42715
--webserver_interface=127.29.112.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab
--principal=kudu/127.29.112.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.212:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.254:42381,127.29.112.253:42521,127.29.112.252:42715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 19:03:30.836817 24877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.824700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35000 (local address 127.29.112.253:42521)
0812 19:03:30.825127 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:30.825147 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:30.825171 (+ 24us) server_negotiation.cc:407] Connection header received
0812 19:03:30.825217 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:30.825222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:30.825437 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:30.825590 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:30.826544 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.827443 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.828498 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:30.829025 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:30.829181 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:30.832067 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:30.832070 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:30.832072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:30.832093 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:30.834190 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:30.834657 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:30.834666 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:30.834671 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:30.834765 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:30.835067 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:30.835070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:30.835071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:30.835415 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:30.835544 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:30.836482 (+ 938us) server_negotiation.cc:299] Negotiation successful
0812 19:03:30.836610 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":294,"thread_start_us":172,"threads_started":1}
I20260812 19:03:30.838733 24782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:03:30.839217 24867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:03:30.839485 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.839177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:41888)
0812 19:03:30.839333 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.839400 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 19:03:30.839951 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.839654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:41894)
0812 19:03:30.839806 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.839880 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260812 19:03:30.840184 24808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
I20260812 19:03:30.840323 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.840044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:41906)
0812 19:03:30.840206 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.840259 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260812 19:03:30.840487 24782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:42715: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
I20260812 19:03:30.841126 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.840934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:41914)
0812 19:03:30.841038 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.841081 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 19:03:30.841343 24867 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:42715: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
I20260812 19:03:30.871659 24782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } attempt: 1
I20260812 19:03:30.872720 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.872310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:41920)
0812 19:03:30.872529 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.872614 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 19:03:30.873651 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.873400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:41934)
0812 19:03:30.873529 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.873584 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 19:03:30.873803 24782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:42715: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
I20260812 19:03:30.890594 24867 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } attempt: 1
I20260812 19:03:30.891433 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.891145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:41950)
0812 19:03:30.891284 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.891345 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 19:03:30.892166 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.891964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:41964)
0812 19:03:30.892068 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.892118 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 19:03:30.892292 24867 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:42715: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
I20260812 19:03:30.949018 24782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } attempt: 2
W20260812 19:03:30.949461 24880 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 19:03:30.949797 24880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 19:03:30.949867 24880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 19:03:30.949920 24880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 19:03:30.949910 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.949528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:41978)
0812 19:03:30.949710 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.949783 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 19:03:30.950657 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.950424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:41980)
0812 19:03:30.950539 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.950594 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 19:03:30.950819 24782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:42715: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
W20260812 19:03:30.954435 24880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 19:03:30.954490 24880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 19:03:30.954510 24880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 19:03:30.954526 24880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 19:03:30.954571 24880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 19:03:30.959890 24880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.212:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.254:42381,127.29.112.253:42521,127.29.112.252:42715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/krb5kdc/kudu_127.29.112.252.keytab
--principal=kudu/127.29.112.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.252:42715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.112.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 18:16:19 UTC on ccycloud.quasar-vfnacu.root.comops.site
build id 4583234
I20260812 19:03:30.960470 24867 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } attempt: 2
I20260812 19:03:30.961002 24880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:03:30.961309 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.961003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:41986)
0812 19:03:30.961157 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.961231 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 19:03:30.962123 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:30.961900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:42000)
0812 19:03:30.962015 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:30.962070 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 19:03:30.962155 24880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:03:30.962252 24867 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.252:42715: Network error: Client connection negotiation failed: client connection to 127.29.112.252:42715: connect: Connection refused (error 111)
W20260812 19:03:30.970513 24886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:30.970825 24887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:03:30.970510 24889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:03:30.971006 24880 server_base.cc:1034] running on GCE node
I20260812 19:03:30.971537 24880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 19:03:30.972791 24880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 19:03:30.973963 24880 hybrid_clock.cc:648] HybridClock initialized: now 1786561410973927 us; error 46 us; skew 500 ppm
Aug 12 19:03:30 dist-test-slave-6nmv krb5kdc[24688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 19:03:30.976903 24880 init.cc:377] Logged in from keytab as kudu/127.29.112.252@KRBTEST.COM (short username kudu)
I20260812 19:03:30.977947 24880 webserver.cc:460] Webserver started at http://127.29.112.252:37627/ using document root <none> and password file <none>
I20260812 19:03:30.978556 24880 fs_manager.cc:359] Metadata directory not provided
I20260812 19:03:30.978655 24880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:03:30.978891 24880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:03:30.981021 24880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/data/instance:
uuid: "489ccfce8e624a2dbaa3258736d68f50"
format_stamp: "Formatted at 2026-08-12 19:03:30 on dist-test-slave-6nmv"
server_key: "df2a541b9b720b3c3399d53907902cef"
server_key_iv: "66beb7cc8a8d1e2ea4792fd77c903b91"
server_key_version: "encryptionkey@0"
I20260812 19:03:30.981622 24880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/wal/instance:
uuid: "489ccfce8e624a2dbaa3258736d68f50"
format_stamp: "Formatted at 2026-08-12 19:03:30 on dist-test-slave-6nmv"
server_key: "df2a541b9b720b3c3399d53907902cef"
server_key_iv: "66beb7cc8a8d1e2ea4792fd77c903b91"
server_key_version: "encryptionkey@0"
I20260812 19:03:30.985977 24880 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 19:03:30.988490 24897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:03:30.989758 24880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 19:03:30.989929 24880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/data,/tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/wal
uuid: "489ccfce8e624a2dbaa3258736d68f50"
format_stamp: "Formatted at 2026-08-12 19:03:30 on dist-test-slave-6nmv"
server_key: "df2a541b9b720b3c3399d53907902cef"
server_key_iv: "66beb7cc8a8d1e2ea4792fd77c903b91"
server_key_version: "encryptionkey@0"
I20260812 19:03:30.990027 24880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:03:31.006982 24880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:03:31.010267 24880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:03:31.010444 24880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:03:31.018285 24880 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.252:42715
I20260812 19:03:31.018594 24953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.252:42715 every 8 connection(s)
I20260812 19:03:31.019450 24880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/data/info.pb
I20260812 19:03:31.022275 24955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:03:31.025247 30147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu as pid 24880
I20260812 19:03:31.025424 30147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmcQeqc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786561125017040-30147-0/minicluster-data/master-2/wal/instance
I20260812 19:03:31.025713 30147 external_mini_cluster.cc:1428] Setting key f5007e31b158211619b3ff132dba06c5
I20260812 19:03:31.026072 24955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:03:31 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
Aug 12 19:03:31 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
I20260812 19:03:31.038735 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.027984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50284 (local address 127.29.112.254:42381)
0812 19:03:31.028146 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.028151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.028410 (+ 259us) server_negotiation.cc:407] Connection header received
0812 19:03:31.028822 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.028825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.028888 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.028979 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.029783 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.030343 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.031447 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.031695 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.031808 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.034681 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:31.034687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:31.034691 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:31.034725 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:31.036347 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.036887 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.036889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.036891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.036966 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.037337 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.037340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.037342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.037612 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:31.037699 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:31.038379 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 19:03:31.038528 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 19:03:31.039716 24959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.027869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.254:42381 (local address 127.0.0.1:50284)
0812 19:03:31.028258 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:31.028298 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:31.028617 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:31.029032 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:31.029038 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:31.029390 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:31.029642 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:31.029656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.030458 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.030463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:31.031318 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:31.031328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.031852 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.031856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:31.032025 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.032658 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:31.032675 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:31.034483 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:31.036455 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:31.036464 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:31.036475 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:31.036790 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:31.037057 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:31.037062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:31.037064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:31.037242 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:31.037717 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:31.037724 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:31.038099 (+ 375us) client_negotiation.cc:769] Sending connection context
0812 19:03:31.038317 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 19:03:31.038658 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":70,"thread_start_us":107,"threads_started":1}
I20260812 19:03:31.039716 24958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.027132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.29.112.252:42715)
0812 19:03:31.027536 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.027572 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.027614 (+ 42us) server_negotiation.cc:407] Connection header received
0812 19:03:31.028589 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.028614 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.028849 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.029059 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.030007 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.030936 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.031899 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.032274 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.032403 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.035142 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:31.035150 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:31.035164 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:31.035194 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:31.037157 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.037634 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.037638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.037639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.037743 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.038058 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.038061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.038063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.038332 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:31.038532 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:31.038762 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 19:03:31.038889 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":127,"threads_started":1}
I20260812 19:03:31.041642 24955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 19:03:31 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
Aug 12 19:03:31 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.254@KRBTEST.COM
I20260812 19:03:31.049953 24782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } attempt: 3
I20260812 19:03:31.050426 24959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.042187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.253:42521 (local address 127.0.0.1:35014)
0812 19:03:31.042525 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:31.042537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:31.042634 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:31.042874 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:31.042877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:31.043047 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:31.043129 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:31.043134 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.043844 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.043851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:31.044342 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:31.044349 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.044858 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.044862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:31.044965 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.045473 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:31.045487 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:31.046765 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:31.048722 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:31.048727 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:31.048729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:31.048991 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:31.049273 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:31.049279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:31.049282 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:31.049380 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:31.049840 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:31.049843 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:31.049947 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 19:03:31.050064 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 19:03:31.050228 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267}
I20260812 19:03:31.050925 24877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.042297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35014 (local address 127.29.112.253:42521)
0812 19:03:31.042473 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.042478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.042599 (+ 121us) server_negotiation.cc:407] Connection header received
0812 19:03:31.042713 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.042717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.042767 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.042863 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.043245 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.043738 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.044464 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.044742 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.044856 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.046920 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:31.046924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:31.046926 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:31.046974 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:31.048609 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.049094 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.049097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.049098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.049180 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.049490 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.049494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.049495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.049711 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:31.049848 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:31.050255 (+ 407us) server_negotiation.cc:299] Negotiation successful
0812 19:03:31.050421 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 19:03:31.051818 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.042159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50288 (local address 127.29.112.254:42381)
0812 19:03:31.042420 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.042423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.042436 (+ 13us) server_negotiation.cc:407] Connection header received
0812 19:03:31.042464 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.042466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.042499 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.042599 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.043185 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.043655 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.044396 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.044639 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.044745 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.047417 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:31.047424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:31.047429 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:31.047460 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:31.049216 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.049631 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.049635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.049637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.049733 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.050310 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.050314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.050315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.050554 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:31.050786 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:31.051509 (+ 723us) server_negotiation.cc:299] Negotiation successful
0812 19:03:31.051668 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
I20260812 19:03:31.051891 24955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } has no permanent_uuid. Determining permanent_uuid...
I20260812 19:03:31.053900 24867 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } attempt: 3
Aug 12 19:03:31 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.254@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
Aug 12 19:03:31 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.252@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
Aug 12 19:03:31 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.253@KRBTEST.COM for kudu/127.29.112.252@KRBTEST.COM
Aug 12 19:03:31 dist-test-slave-6nmv krb5kdc[24688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786561410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.253@KRBTEST.COM
I20260812 19:03:31.063503 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.050399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:42010)
0812 19:03:31.050698 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:31.050712 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:31.050785 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:31.051224 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:31.051227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:31.051425 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:31.051531 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:31.051540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.052659 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.052663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:31.053436 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:31.053448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.054103 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.054107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:31.054210 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.055040 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:31.055059 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:31.056834 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:31.061323 (+ 4489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:31.061328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:31.061331 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:31.061675 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:31.062087 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:31.062091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:31.062094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:31.062221 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:31.062796 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:31.062800 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:31.062938 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 19:03:31.063052 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 19:03:31.063251 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211}
I20260812 19:03:31.064113 24958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.050900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42010 (local address 127.29.112.252:42715)
0812 19:03:31.051024 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.051028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.051040 (+ 12us) server_negotiation.cc:407] Connection header received
0812 19:03:31.051073 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.051076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.051128 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.051203 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.051693 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.052487 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.053571 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.053962 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.054469 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.056993 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:31.056997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:31.056999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:31.057026 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:31.061173 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.061847 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.061851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.061852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.061944 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.062365 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.062368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.062370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.062641 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:31.062782 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:31.063766 (+ 984us) server_negotiation.cc:299] Negotiation successful
0812 19:03:31.063901 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:03:31.066063 24959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.052578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:42018)
0812 19:03:31.053270 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:31.053286 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:31.053420 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:31.053729 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:31.053732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:31.053955 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:31.054120 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:31.054128 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.056131 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.056134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:31.056993 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:31.057000 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.058065 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.058068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:31.058143 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.058596 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:31.058614 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:31.060785 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:31.063852 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:31.063856 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:31.063858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:31.064392 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:31.064685 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:31.064688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:31.064689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:31.064769 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:31.065478 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:31.065481 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:31.065575 (+ 94us) client_negotiation.cc:769] Sending connection context
0812 19:03:31.065691 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 19:03:31.065878 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382}
I20260812 19:03:31.066749 24960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.052661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.29.112.252:42715)
0812 19:03:31.053053 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.053071 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.053354 (+ 283us) server_negotiation.cc:407] Connection header received
0812 19:03:31.053523 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.053529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.053574 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.053646 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.054226 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.055984 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.057114 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.057844 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.057944 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.060954 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:31.060960 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:31.060967 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:31.060998 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:31.063722 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.064505 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.064507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.064509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.064592 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.065141 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.065143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.065145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.065387 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:31.065479 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:31.066440 (+ 961us) server_negotiation.cc:299] Negotiation successful
0812 19:03:31.066575 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":121,"threads_started":1}
I20260812 19:03:31.067076 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.054306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.252:42715 (local address 127.0.0.1:42030)
0812 19:03:31.054483 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 19:03:31.054497 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 19:03:31.054559 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 19:03:31.055899 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 19:03:31.055902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 19:03:31.056136 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0812 19:03:31.056234 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:31.056241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.057615 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.057619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:31.058488 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 19:03:31.058501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.059669 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.059675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 19:03:31.059793 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.060317 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 19:03:31.060340 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 19:03:31.062228 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 19:03:31.065256 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:31.065260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:31.065262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:31.065564 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:31.065957 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 19:03:31.065961 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 19:03:31.065963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 19:03:31.066058 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 19:03:31.066504 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 19:03:31.066508 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 19:03:31.066618 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 19:03:31.066717 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 19:03:31.066878 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"spinlock_wait_cycles":6016}
I20260812 19:03:31.067649 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.054758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42030 (local address 127.29.112.252:42715)
0812 19:03:31.055345 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.055354 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.055368 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:03:31.055658 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.055661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.055715 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.056404 (+ 689us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.056544 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.057437 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.059156 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.059523 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.059635 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.062401 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:31.062406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:31.062411 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:31.062443 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:31.065132 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.065690 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.065693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.065695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.065774 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.066189 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.066193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.066197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.066391 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:31.067363 (+ 972us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:31.067444 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 19:03:31.067534 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":154,"threads_started":1}
I20260812 19:03:31.068272 24877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.053761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35016 (local address 127.29.112.253:42521)
0812 19:03:31.053929 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.053934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.054260 (+ 326us) server_negotiation.cc:407] Connection header received
0812 19:03:31.054301 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.054304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.054351 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.054421 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.055928 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.056712 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.059212 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.059537 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.059634 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.063792 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:31.063796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:31.063799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:31.063825 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:31.066082 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.066627 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.066634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.066636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.066742 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.067241 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.067244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.067246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.067459 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:31.067675 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:31.068059 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 19:03:31.068163 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63,"spinlock_wait_cycles":896}
I20260812 19:03:31.068427 24782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af: Bootstrap starting.
I20260812 19:03:31.071486 24782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:31.072352 24782 log.cc:824] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:31.073133 24867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3: Bootstrap starting.
I20260812 19:03:31.073133 24955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50: Bootstrap starting.
I20260812 19:03:31.074760 24782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af: No bootstrap required, opened a new log
I20260812 19:03:31.076097 24867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:31.076110 24955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50: Neither blocks nor log segments found. Creating new log.
I20260812 19:03:31.077065 24867 log.cc:824] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:31.077064 24955 log.cc:824] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50: Log is configured to *not* fsync() on all Append() calls
I20260812 19:03:31.079643 24867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3: No bootstrap required, opened a new log
I20260812 19:03:31.079782 24955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50: No bootstrap required, opened a new log
I20260812 19:03:31.080127 30147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 19:03:31.080430 24782 raft_consensus.cc:348] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } }
I20260812 19:03:31.080844 24782 raft_consensus.cc:729] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f19b6157d3cb442a83e81700c34260af, State: Initialized, Role: FOLLOWER
I20260812 19:03:31.081223 24782 consensus_queue.cc:260] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } }
I20260812 19:03:31.081509 24754 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f19b6157d3cb442a83e81700c34260af"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50288
I20260812 19:03:31.082871 24782 sys_catalog.cc:564] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:31.082948 24754 raft_consensus.cc:482] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:03:31.083098 24754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:31.084909 24754 raft_consensus.cc:504] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } }
I20260812 19:03:31.084975 24867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } }
I20260812 19:03:31.085230 24867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33a73d89f2394f7bbfe15b5b6e23b4b3, State: Initialized, Role: FOLLOWER
I20260812 19:03:31.085580 24867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } }
I20260812 19:03:31.086050 24955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } }
I20260812 19:03:31.086388 24955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 489ccfce8e624a2dbaa3258736d68f50, State: Initialized, Role: FOLLOWER
I20260812 19:03:31.086519 24754 leader_election.cc:288] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [CANDIDATE]: Term 1 election: Requested vote from peers 33a73d89f2394f7bbfe15b5b6e23b4b3 (127.29.112.253:42521), 489ccfce8e624a2dbaa3258736d68f50 (127.29.112.252:42715)
I20260812 19:03:31.086833 24955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } }
I20260812 19:03:31.087141 24867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:31.088101 24838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f19b6157d3cb442a83e81700c34260af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3"
I20260812 19:03:31.088359 24838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:31.089289 24926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f19b6157d3cb442a83e81700c34260af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "489ccfce8e624a2dbaa3258736d68f50"
I20260812 19:03:31.089529 24926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:03:31.090044 24838 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f19b6157d3cb442a83e81700c34260af in term 1.
I20260812 19:03:31.090648 24723 leader_election.cc:302] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33a73d89f2394f7bbfe15b5b6e23b4b3, f19b6157d3cb442a83e81700c34260af; no voters:
I20260812 19:03:31.091049 24926 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f19b6157d3cb442a83e81700c34260af in term 1.
I20260812 19:03:31.093510 24955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:03:31.096138 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } } }
W20260812 19:03:31.096319 24980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 19:03:31.096521 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [sys.catalog]: This master's current role is: FOLLOWER
W20260812 19:03:31.096550 24980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:31.096521 24969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:03:31.098023 24981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } } }
I20260812 19:03:31.098145 24969 raft_consensus.cc:686] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [term 1 LEADER]: Becoming Leader. State: Replica: f19b6157d3cb442a83e81700c34260af, State: Running, Role: LEADER
I20260812 19:03:31.098198 24981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:31.098506 24969 consensus_queue.cc:237] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } }
I20260812 19:03:31.098457 24982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } } }
I20260812 19:03:31.098764 24982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:31.093113 24704 master_runner.cc:186] Error getting master registration for 127.29.112.253:42521: OK, instance_id { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" instance_seqno: 1786561410794465 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 19:03:31.099061 24704 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:03:31.101344 24790 master_runner.cc:263] No leader master found from master 33a73d89f2394f7bbfe15b5b6e23b4b3
I20260812 19:03:31.101415 24790 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 19:03:31.102088 24969 sys_catalog.cc:455] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [sys.catalog]: SysCatalogTable state changed. Reason: New leader f19b6157d3cb442a83e81700c34260af. Latest consensus state: current_term: 1 leader_uuid: "f19b6157d3cb442a83e81700c34260af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } } }
I20260812 19:03:31.102223 24969 sys_catalog.cc:458] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [sys.catalog]: This master's current role is: LEADER
W20260812 19:03:31.104343 25019 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:03:31.105854 25019 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:31.114624 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.089762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.29.112.252:42715)
0812 19:03:31.091792 (+ 2030us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.091797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.099638 (+ 7841us) server_negotiation.cc:407] Connection header received
0812 19:03:31.100851 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.100855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.100911 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.100975 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.105689 (+ 4714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.106518 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.107354 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.107765 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.107873 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.110002 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:31.110007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:31.110010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:31.110040 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:31.112301 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.112849 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.112853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.112855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.112985 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.113339 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.113343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.113345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.113578 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:31.113720 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:31.114229 (+ 509us) server_negotiation.cc:299] Negotiation successful
0812 19:03:31.114391 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1871}
I20260812 19:03:31.115203 25022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:03:31.116214 25022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:03:31.121749 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.088975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50298 (local address 127.29.112.254:42381)
0812 19:03:31.089123 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.089129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.091368 (+ 2239us) server_negotiation.cc:407] Connection header received
0812 19:03:31.091667 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.091671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.091735 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.091799 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.103624 (+ 11825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.104408 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.109617 (+ 5209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.110052 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.110140 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.117498 (+ 7358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:31.117502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:31.117505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:31.117537 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:31.119538 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.120096 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.120101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.120103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.120202 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.120565 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.120569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.120571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.120827 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:31.120955 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:31.121351 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 19:03:31.121539 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 19:03:31.122094 24877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:31.090430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35020 (local address 127.29.112.253:42521)
0812 19:03:31.091355 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.091359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.102261 (+ 10902us) server_negotiation.cc:407] Connection header received
0812 19:03:31.102462 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.102465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.102521 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.102576 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.107410 (+ 4834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.108223 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.114875 (+ 6652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:31.115245 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:31.115364 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:31.117441 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 19:03:31.117445 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 19:03:31.117448 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 19:03:31.117480 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 19:03:31.119952 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.120486 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.120493 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.120496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.120595 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 19:03:31.120946 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 19:03:31.120950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 19:03:31.120952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 19:03:31.121185 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 19:03:31.121326 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:31.121723 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 19:03:31.121903 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660}
W20260812 19:03:31.123917 25004 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 19:03:31.124007 25004 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 19:03:31.133513 24838 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [term 1 FOLLOWER]: Refusing update from remote peer f19b6157d3cb442a83e81700c34260af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:31.133548 24926 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [term 1 FOLLOWER]: Refusing update from remote peer f19b6157d3cb442a83e81700c34260af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:03:31.134225 24969 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [LEADER]: Connected to new peer: Peer: permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:31.134644 24969 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [LEADER]: Connected to new peer: Peer: permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:03:31.138784 24981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f19b6157d3cb442a83e81700c34260af. Latest consensus state: current_term: 1 leader_uuid: "f19b6157d3cb442a83e81700c34260af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } } }
I20260812 19:03:31.138913 24981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:31.143412 24982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f19b6157d3cb442a83e81700c34260af. Latest consensus state: current_term: 1 leader_uuid: "f19b6157d3cb442a83e81700c34260af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } } }
I20260812 19:03:31.143564 24982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:31.146759 25024 mvcc.cc:205] Tried to move back new op lower bound from 7317755539993788416 to 7317755539863633920. Current Snapshot: MvccSnapshot[applied={T|T < 7317755539993788416}]
I20260812 19:03:31.147320 24969 sys_catalog.cc:455] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f19b6157d3cb442a83e81700c34260af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } } }
I20260812 19:03:31.147452 24969 sys_catalog.cc:458] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:31.149526 24981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f19b6157d3cb442a83e81700c34260af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } } }
I20260812 19:03:31.149652 24981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:31.151144 25022 catalog_manager.cc:1347] Generated new cluster ID: e4f98d5d801f425084e65944889967f2
I20260812 19:03:31.151216 25022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:03:31.153918 25026 mvcc.cc:205] Tried to move back new op lower bound from 7317755539993788416 to 7317755539863633920. Current Snapshot: MvccSnapshot[applied={T|T < 7317755539993788416}]
I20260812 19:03:31.154882 25025 sys_catalog.cc:455] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f19b6157d3cb442a83e81700c34260af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } } }
I20260812 19:03:31.155005 25025 sys_catalog.cc:458] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af [sys.catalog]: This master's current role is: LEADER
I20260812 19:03:31.156054 24982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f19b6157d3cb442a83e81700c34260af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b6157d3cb442a83e81700c34260af" member_type: VOTER last_known_addr { host: "127.29.112.254" port: 42381 } } peers { permanent_uuid: "33a73d89f2394f7bbfe15b5b6e23b4b3" member_type: VOTER last_known_addr { host: "127.29.112.253" port: 42521 } } peers { permanent_uuid: "489ccfce8e624a2dbaa3258736d68f50" member_type: VOTER last_known_addr { host: "127.29.112.252" port: 42715 } } }
I20260812 19:03:31.156164 24982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 19:03:31.156582 25031 mvcc.cc:205] Tried to move back new op lower bound from 7317755539993788416 to 7317755539863633920. Current Snapshot: MvccSnapshot[applied={T|T < 7317755539993788416}]
I20260812 19:03:31.171540 25022 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:03:31.173672 25022 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:03:31.193684 25022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f19b6157d3cb442a83e81700c34260af: Generated new TSK 0
I20260812 19:03:31.194864 25022 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 19:03:31.228261 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.225196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.29.112.253:42521)
0812 19:03:31.225321 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.225325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.227170 (+ 1845us) server_negotiation.cc:407] Connection header received
0812 19:03:31.227373 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.227376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.227432 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.227485 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.228131 (+ 646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35032: BlockingRecv error: recv got EOF from 127.0.0.1:35032 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 19:03:31.229542 24786 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.225674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50308 (local address 127.29.112.254:42381)
0812 19:03:31.225782 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.225786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.228719 (+ 2933us) server_negotiation.cc:407] Connection header received
0812 19:03:31.228879 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.228883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.228929 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.228975 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.229430 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50308: BlockingRecv error: recv got EOF from 127.0.0.1:50308 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 19:03:31.230912 24961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.226659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.29.112.252:42715)
0812 19:03:31.226800 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.226805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.230017 (+ 3212us) server_negotiation.cc:407] Connection header received
0812 19:03:31.230186 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.230189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.230237 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.230304 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.230796 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42036: BlockingRecv error: recv got EOF from 127.0.0.1:42036 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 19:03:31.235296 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.233714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.29.112.253:42521)
0812 19:03:31.233813 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.233817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.234328 (+ 511us) server_negotiation.cc:407] Connection header received
0812 19:03:31.234557 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.234561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.234640 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.234691 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.235201 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35034: BlockingRecv error: recv got EOF from 127.0.0.1:35034 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 19:03:31.240124 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.238830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35036 (local address 127.29.112.253:42521)
0812 19:03:31.238951 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.238955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.239273 (+ 318us) server_negotiation.cc:407] Connection header received
0812 19:03:31.239464 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.239467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.239518 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.239567 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.240032 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35036: BlockingRecv error: recv got EOF from 127.0.0.1:35036 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 19:03:31.245613 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.244136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.29.112.253:42521)
0812 19:03:31.244229 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.244232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.244771 (+ 539us) server_negotiation.cc:407] Connection header received
0812 19:03:31.244958 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.244962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.245019 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.245067 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.245523 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35046: BlockingRecv error: recv got EOF from 127.0.0.1:35046 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 19:03:31.254477 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.251819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35052 (local address 127.29.112.253:42521)
0812 19:03:31.251999 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.252004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.253445 (+ 1441us) server_negotiation.cc:407] Connection header received
0812 19:03:31.253649 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.253653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.253704 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.253758 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.254365 (+ 607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35052: BlockingRecv error: recv got EOF from 127.0.0.1:35052 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260812 19:03:31.266629 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.264853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35058 (local address 127.29.112.253:42521)
0812 19:03:31.264971 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.264976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.265564 (+ 588us) server_negotiation.cc:407] Connection header received
0812 19:03:31.265770 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.265774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.265861 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.265919 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.266514 (+ 595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35058: BlockingRecv error: recv got EOF from 127.0.0.1:35058 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 19:03:31.288113 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.285886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35066 (local address 127.29.112.253:42521)
0812 19:03:31.286020 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.286026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.286795 (+ 769us) server_negotiation.cc:407] Connection header received
0812 19:03:31.287069 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.287073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.287140 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.287207 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.287988 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35066: BlockingRecv error: recv got EOF from 127.0.0.1:35066 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 19:03:31.325002 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.323191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35078 (local address 127.29.112.253:42521)
0812 19:03:31.323319 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.323323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.323927 (+ 604us) server_negotiation.cc:407] Connection header received
0812 19:03:31.324137 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.324141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.324196 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.324266 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.324876 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35078: BlockingRecv error: recv got EOF from 127.0.0.1:35078 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 19:03:31.393011 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.391780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35084 (local address 127.29.112.253:42521)
0812 19:03:31.391946 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.391951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.392120 (+ 169us) server_negotiation.cc:407] Connection header received
0812 19:03:31.392228 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.392231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.392280 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.392354 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.392836 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35084: BlockingRecv error: recv got EOF from 127.0.0.1:35084 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 19:03:31.524407 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.523303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35100 (local address 127.29.112.253:42521)
0812 19:03:31.523444 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.523448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.523540 (+ 92us) server_negotiation.cc:407] Connection header received
0812 19:03:31.523664 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.523667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.523715 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.523804 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.524305 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35100: BlockingRecv error: recv got EOF from 127.0.0.1:35100 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 19:03:31.784004 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 19:03:31.782893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35106 (local address 127.29.112.253:42521)
0812 19:03:31.783051 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:31.783055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:31.783087 (+ 32us) server_negotiation.cc:407] Connection header received
0812 19:03:31.783242 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:31.783245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:31.783294 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:31.783368 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 19:03:31.783845 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35106: BlockingRecv error: recv got EOF from 127.0.0.1:35106 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 19:03:32.107249 25019 catalog_manager.cc:1259] Loaded cluster ID: e4f98d5d801f425084e65944889967f2
I20260812 19:03:32.107384 25019 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50: loading cluster ID for follower catalog manager: success
I20260812 19:03:32.109105 25019 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50: acquiring CA information for follower catalog manager: success
I20260812 19:03:32.110368 25019 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 489ccfce8e624a2dbaa3258736d68f50: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:03:32.125384 25004 catalog_manager.cc:1259] Loaded cluster ID: e4f98d5d801f425084e65944889967f2
I20260812 19:03:32.125474 25004 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3: loading cluster ID for follower catalog manager: success
I20260812 19:03:32.127808 25004 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3: acquiring CA information for follower catalog manager: success
I20260812 19:03:32.128978 25004 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33a73d89f2394f7bbfe15b5b6e23b4b3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 19:03:32.303007 25115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:32.298427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35108 (local address 127.29.112.253:42521)
0812 19:03:32.298685 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:32.298694 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:32.298761 (+ 67us) server_negotiation.cc:407] Connection header received
0812 19:03:32.298933 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:32.298935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:32.299067 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:32.299165 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:32.299490 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:32.300009 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:32.301395 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:32.301625 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:32.301705 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:32.302124 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:32.302577 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:32.302674 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:32.302786 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 19:03:32.302835 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
I20260812 19:03:32.311300 25122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:32.306689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42040 (local address 127.29.112.252:42715)
0812 19:03:32.306961 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:32.306964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:32.307116 (+ 152us) server_negotiation.cc:407] Connection header received
0812 19:03:32.307305 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:32.307307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:32.307426 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:32.307535 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:32.307799 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:32.308306 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:32.309663 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:32.309972 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:32.310064 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:32.310431 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:32.310854 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:32.310949 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:32.311056 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 19:03:32.311110 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":107,"threads_started":1}
I20260812 19:03:32.319342 25115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:32.314847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35118 (local address 127.29.112.253:42521)
0812 19:03:32.314975 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:32.314982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:32.315400 (+ 418us) server_negotiation.cc:407] Connection header received
0812 19:03:32.315491 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:32.315497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:32.315563 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:32.315651 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:32.316000 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:32.316759 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:32.317986 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:32.318325 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:32.318405 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:32.318765 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:32.318912 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:32.318962 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:32.319135 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 19:03:32.319208 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 19:03:32.320212 25122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:32.315477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42050 (local address 127.29.112.252:42715)
0812 19:03:32.315615 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:32.315620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:32.315693 (+ 73us) server_negotiation.cc:407] Connection header received
0812 19:03:32.315810 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:32.315813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:32.315867 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:32.315943 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:32.316366 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:32.317234 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:32.318727 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:32.319114 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:32.319204 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:32.319660 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:32.319752 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:32.319803 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:32.319990 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 19:03:32.320055 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 19:03:32.320575 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 19:03:32.314914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50322 (local address 127.29.112.254:42381)
0812 19:03:32.315674 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0812 19:03:32.315678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 19:03:32.315692 (+ 14us) server_negotiation.cc:407] Connection header received
0812 19:03:32.315788 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 19:03:32.315790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 19:03:32.315960 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 19:03:32.316076 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 19:03:32.316450 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:32.317218 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:32.318404 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 19:03:32.318752 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 19:03:32.318851 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 19:03:32.319289 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 19:03:32.319891 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 19:03:32.319970 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 19:03:32.320354 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 19:03:32.320424 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":111,"threads_started":1}
I20260812 19:03:32.323999 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 24704
I20260812 19:03:32.337085 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 24790
I20260812 19:03:32.346928 30147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmcQeqc/build/debug/bin/kudu with pid 24880
2026-08-12T19:03:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3212 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6473 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (287326 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST